Depression
Conditions
Keywords
workforce development, intelligent tutoring systems
Brief summary
This study will compare training as usual to automated training using an intelligent tutoring system in training bachelors (BA) level social workers in cognitive behavioral therapy (CBT). The purpose of the study is to determine if time and cost of training front line clinicians in evidence-based treatments can be shortened, and if this new training model can reduce the need for clinicians to seek advice from experts.
Detailed description
The University of Washington (UW) School of Social Work, in partnership with Heritage University's School of Social Work in Yakima Valley recently partnered to develop a training program for bachelors (BA) level Social Workers to address limited clinician capacity in rural primary care settings. Currently the curriculum is a combination of didactic training in telephone based cognitive behavioral therapy (CBT; 20 hours), role play training (25 hours), and guided supervision. The scalability of these programs is limited, however, by expert time to conduct training activities, clinician time away from work to engage in training activities, and the fact that even when clinicians participate in training, there is no guarantee they will certify. Adaptive learning, an educational method that uses adaptive algorithms to may be a potential solution to these problems in capacity building. These programs can tailor the educational experience to the needs of the trainee, reduce time in training, improve competence in complex decision-making and standardize training. This study builds on the existing research base on clinical training, and adds to it by designing and testing an intelligent tutoring system (ITS) based on adaptive learning algorithms. Both CBT experts (Aisenberg) and past CBT trainees (Heritage University School of Social Work) will partner with experts in educational software development (Popovic) to create the ITS, which will be compared to training as usual on time to training, competence and skill drift. The investigators hypothesize that capacity building through improved learnability (target mechanism) will result in enhanced clinical ability to deliver CBT elements competently, and in a shorter time-period, and that greater competence will result in better quality of care.
Interventions
Training using didactics and role plays
Training using computerized adaptive training in addition to role play
Sponsors
Study design
Intervention model description
Randomized clinical trial
Eligibility
Inclusion criteria
* BA level social work student * Bilingual Spanish Speaking
Exclusion criteria
* none
Design outcomes
Primary
| Measure | Time frame | Description |
|---|---|---|
| Acceptability of Intervention Measure (AIM) - Goal Setting | This measure will be administered to participants in each training group after they complete the 25 hours of training. | This is a four item measure of intervention acceptability; where each item is rated on a 1-5 scale, with 1 = not at all appropriate and 5 = very appropriate. The total scale range is 4-20. Each group will be compared on the degree they find the specific clinical skill acceptable to use after they complete the training. We administered the AIM twice asking the students to respond to the acceptability of goal setting as a clinical skill and of identifying challenges as a clinical skill. This section reports on goal setting. |
| Acceptability of Intervention Measure (AIM) - Identifying Challenges | This measure will be administered to participants in each training group after they complete the 25 hours of training. | This is a four item measure of intervention acceptability; where each item is rated on a 1-5 scale, with 1 = not at all appropriate and 5 = very appropriate. The total scale range is 4-20. Each group will be compared on the degree they find the specific clinical skill acceptable to use after they complete the training. We administered the AIM twice asking the students to respond to the acceptability of goal setting as a clinical skill and of identifying challenges as a clinical skill. This section reports on identifying challenges. |
| Intervention Appropriateness Measure (IAM) - Goal Setting | This measure will be administered to participants in each training group after they complete the 25 hours of training. | This is a four item measure of intervention appropriateness; where each item is rated on a 1-5 scale, with 1 = not at all appropriate and 5 = very appropriate. The total range for this measure is 4-20. Each group will be compared on the degree they find the specific clinical skill appropriate to use after they complete the training. We administered the IAM twice asking the students to respond to the acceptability of goal setting as a clinical skill and of identifying challenges as a clinical skill. This section reports on goal setting. |
| Intervention Appropriateness Measure (IAM) - Identifying Challenges | This measure will be administered to participants in each training group after they complete the 25 hours of training. | This is a four item measure of intervention appropriateness; where each item is rated on a 1-5 scale, with 1 = not at all appropriate and 5 = very appropriate. The total scale range is 4-20. Each group will be compared on the degree they find the specific clinical skill appropriate to use after they complete the training. We administered the IAM twice asking the students to respond to the acceptability of goal setting as a clinical skill and of identifying challenges as a clinical skill. This section reports on identifying challenges. |
| Feasibility of Intervention Measure (FIM) - Goal Setting | This measure will be administered to participants in each training group after they complete the 25 hours of training. | This is a four item measure of intervention feasibility; where each item is rated on a 1-5 scale, with 1 = not at all appropriate and 5 = very appropriate. The total scale range is 4-20. Each group will be compared on the degree they find the specific clinical skill feasible to use after they complete the training they participated in. We administered the FIM twice asking the students to respond to the feasibility of goal setting as a clinical skill and of identifying challenges as a clinical skill. This section reports on goal setting. |
| Feasibility of Intervention Measure - Identifying Challenges | This measure will be administered to participants in each training group after they complete the 25 hours of training. | This is a four item measure of intervention feasibility; where each item is rated on a 1-5 scale, with 1 = not at all appropriate and 5 = very appropriate. The total scale range is 4-20. Each group will be compared on the degree they find the specific clinical skill feasible to use after they complete the training they participated in. We administered the FIM twice asking the students to respond to the feasibility of goal setting as a clinical skill and of identifying challenges as a clinical skill. This section reports on identifying challenges. |
| Intervention Usability Scale (IUS) - Goal Setting | This measure will be administered to participants in each training group after they complete the 25 hours of training. | The IUS is a 10-item measure with a possible total score ranging from 0 to 100; higher scores indicate a more usable intervention. Although this measure has not yet been normed as a measure of intervention usability, the System Usability Scale upon which the IUS is based defines scores of 70 or above as indicative of acceptable usability. |
| Intervention Usability Scale (IUS) - Identifying Challenges | This measure will be administered to participants in each training group after they complete the 25 hours of training. | The IUS is a 10-item measure with a possible total score ranging from 0 to 100; higher scores indicate a more usable intervention. Although this measure has not yet been normed as a measure of intervention usability, the System Usability Scale upon which the IUS is based defines scores of 70 or above as indicative of acceptable usability. |
Countries
United States
Participant flow
Participants by arm
| Arm | Count |
|---|---|
| Traditional Training 24 hours of didactic and simulated case role play training in CBT principles, depression assessment and cultural competency.
Traditional Training: Training using didactics and role plays | 8 |
| ITS Based Training Traditional training plus the addition of an algorithmic based training computer program that trains clinicians in clinical micro-competencies.
ITS based training: Training using computerized adaptive training in addition to role play | 11 |
| Total | 19 |
Baseline characteristics
| Characteristic | ITS Based Training | Total | Traditional Training |
|---|---|---|---|
| Age, Continuous | 24.09 years STANDARD_DEVIATION 8.63 | 24.74 years STANDARD_DEVIATION 8.82 | 26.63 years STANDARD_DEVIATION 9.61 |
| Ethnicity (NIH/OMB) Hispanic or Latino | 4 Participants | 12 Participants | 8 Participants |
| Ethnicity (NIH/OMB) Not Hispanic or Latino | 7 Participants | 7 Participants | 0 Participants |
| Ethnicity (NIH/OMB) Unknown or Not Reported | 0 Participants | 0 Participants | 0 Participants |
| Race (NIH/OMB) American Indian or Alaska Native | 0 Participants | 0 Participants | 0 Participants |
| Race (NIH/OMB) Asian | 1 Participants | 1 Participants | 0 Participants |
| Race (NIH/OMB) Black or African American | 1 Participants | 1 Participants | 0 Participants |
| Race (NIH/OMB) More than one race | 1 Participants | 1 Participants | 0 Participants |
| Race (NIH/OMB) Native Hawaiian or Other Pacific Islander | 0 Participants | 0 Participants | 0 Participants |
| Race (NIH/OMB) Unknown or Not Reported | 0 Participants | 0 Participants | 0 Participants |
| Race (NIH/OMB) White | 8 Participants | 16 Participants | 8 Participants |
| Sex: Female, Male Female | 9 Participants | 17 Participants | 8 Participants |
| Sex: Female, Male Male | 2 Participants | 2 Participants | 0 Participants |
Adverse events
| Event type | EG000 affected / at risk | EG001 affected / at risk |
|---|---|---|
| deaths Total, all-cause mortality | 0 / 8 | 0 / 11 |
| other Total, other adverse events | 0 / 8 | 0 / 11 |
| serious Total, serious adverse events | 0 / 8 | 0 / 11 |
Outcome results
Acceptability of Intervention Measure (AIM) - Goal Setting
This is a four item measure of intervention acceptability; where each item is rated on a 1-5 scale, with 1 = not at all appropriate and 5 = very appropriate. The total scale range is 4-20. Each group will be compared on the degree they find the specific clinical skill acceptable to use after they complete the training. We administered the AIM twice asking the students to respond to the acceptability of goal setting as a clinical skill and of identifying challenges as a clinical skill. This section reports on goal setting.
Time frame: This measure will be administered to participants in each training group after they complete the 25 hours of training.
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Traditional Training | Acceptability of Intervention Measure (AIM) - Goal Setting | Pre-Training | 17.88 score on a scale | Standard Deviation 0.99 |
| Traditional Training | Acceptability of Intervention Measure (AIM) - Goal Setting | Post-Training | 17.63 score on a scale | Standard Deviation 1.69 |
| ITS Based Training | Acceptability of Intervention Measure (AIM) - Goal Setting | Pre-Training | 17.64 score on a scale | Standard Deviation 1.43 |
| ITS Based Training | Acceptability of Intervention Measure (AIM) - Goal Setting | Post-Training | 18.27 score on a scale | Standard Deviation 2.61 |
Acceptability of Intervention Measure (AIM) - Identifying Challenges
This is a four item measure of intervention acceptability; where each item is rated on a 1-5 scale, with 1 = not at all appropriate and 5 = very appropriate. The total scale range is 4-20. Each group will be compared on the degree they find the specific clinical skill acceptable to use after they complete the training. We administered the AIM twice asking the students to respond to the acceptability of goal setting as a clinical skill and of identifying challenges as a clinical skill. This section reports on identifying challenges.
Time frame: This measure will be administered to participants in each training group after they complete the 25 hours of training.
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Traditional Training | Acceptability of Intervention Measure (AIM) - Identifying Challenges | Pre-Training | 16.75 score on a scale | Standard Deviation 1.39 |
| Traditional Training | Acceptability of Intervention Measure (AIM) - Identifying Challenges | Post-Training | 16.25 score on a scale | Standard Deviation 1.28 |
| ITS Based Training | Acceptability of Intervention Measure (AIM) - Identifying Challenges | Pre-Training | 17.73 score on a scale | Standard Deviation 3.44 |
| ITS Based Training | Acceptability of Intervention Measure (AIM) - Identifying Challenges | Post-Training | 17.82 score on a scale | Standard Deviation 1.72 |
Feasibility of Intervention Measure (FIM) - Goal Setting
This is a four item measure of intervention feasibility; where each item is rated on a 1-5 scale, with 1 = not at all appropriate and 5 = very appropriate. The total scale range is 4-20. Each group will be compared on the degree they find the specific clinical skill feasible to use after they complete the training they participated in. We administered the FIM twice asking the students to respond to the feasibility of goal setting as a clinical skill and of identifying challenges as a clinical skill. This section reports on goal setting.
Time frame: This measure will be administered to participants in each training group after they complete the 25 hours of training.
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Traditional Training | Feasibility of Intervention Measure (FIM) - Goal Setting | Pre-Training | 16.88 score on a scale | Standard Deviation 2.23 |
| Traditional Training | Feasibility of Intervention Measure (FIM) - Goal Setting | Post-Training | 17.13 score on a scale | Standard Deviation 1.46 |
| ITS Based Training | Feasibility of Intervention Measure (FIM) - Goal Setting | Post-Training | 18.91 score on a scale | Standard Deviation 1.64 |
| ITS Based Training | Feasibility of Intervention Measure (FIM) - Goal Setting | Pre-Training | 17.09 score on a scale | Standard Deviation 2.17 |
Feasibility of Intervention Measure - Identifying Challenges
This is a four item measure of intervention feasibility; where each item is rated on a 1-5 scale, with 1 = not at all appropriate and 5 = very appropriate. The total scale range is 4-20. Each group will be compared on the degree they find the specific clinical skill feasible to use after they complete the training they participated in. We administered the FIM twice asking the students to respond to the feasibility of goal setting as a clinical skill and of identifying challenges as a clinical skill. This section reports on identifying challenges.
Time frame: This measure will be administered to participants in each training group after they complete the 25 hours of training.
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Traditional Training | Feasibility of Intervention Measure - Identifying Challenges | Pre-Training | 15.88 score on a scale | Standard Deviation 2.3 |
| Traditional Training | Feasibility of Intervention Measure - Identifying Challenges | Post-Training | 15.63 score on a scale | Standard Deviation 1.69 |
| ITS Based Training | Feasibility of Intervention Measure - Identifying Challenges | Pre-Training | 17.73 score on a scale | Standard Deviation 2.45 |
| ITS Based Training | Feasibility of Intervention Measure - Identifying Challenges | Post-Training | 18.27 score on a scale | Standard Deviation 1.68 |
Intervention Appropriateness Measure (IAM) - Goal Setting
This is a four item measure of intervention appropriateness; where each item is rated on a 1-5 scale, with 1 = not at all appropriate and 5 = very appropriate. The total range for this measure is 4-20. Each group will be compared on the degree they find the specific clinical skill appropriate to use after they complete the training. We administered the IAM twice asking the students to respond to the acceptability of goal setting as a clinical skill and of identifying challenges as a clinical skill. This section reports on goal setting.
Time frame: This measure will be administered to participants in each training group after they complete the 25 hours of training.
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Traditional Training | Intervention Appropriateness Measure (IAM) - Goal Setting | Pre-Training | 18.25 score on a scale | Standard Deviation 1.75 |
| Traditional Training | Intervention Appropriateness Measure (IAM) - Goal Setting | Post-Training | 17.88 score on a scale | Standard Deviation 2.23 |
| ITS Based Training | Intervention Appropriateness Measure (IAM) - Goal Setting | Post-Training | 18.82 score on a scale | Standard Deviation 1.54 |
| ITS Based Training | Intervention Appropriateness Measure (IAM) - Goal Setting | Pre-Training | 18.55 score on a scale | Standard Deviation 1.63 |
Intervention Appropriateness Measure (IAM) - Identifying Challenges
This is a four item measure of intervention appropriateness; where each item is rated on a 1-5 scale, with 1 = not at all appropriate and 5 = very appropriate. The total scale range is 4-20. Each group will be compared on the degree they find the specific clinical skill appropriate to use after they complete the training. We administered the IAM twice asking the students to respond to the acceptability of goal setting as a clinical skill and of identifying challenges as a clinical skill. This section reports on identifying challenges.
Time frame: This measure will be administered to participants in each training group after they complete the 25 hours of training.
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Traditional Training | Intervention Appropriateness Measure (IAM) - Identifying Challenges | Pre-Training | 16.38 score on a scale | Standard Deviation 1.19 |
| Traditional Training | Intervention Appropriateness Measure (IAM) - Identifying Challenges | Post-Training | 17.13 score on a scale | Standard Deviation 1.81 |
| ITS Based Training | Intervention Appropriateness Measure (IAM) - Identifying Challenges | Pre-Training | 18.82 score on a scale | Standard Deviation 2.27 |
| ITS Based Training | Intervention Appropriateness Measure (IAM) - Identifying Challenges | Post-Training | 18.27 score on a scale | Standard Deviation 1.9 |
Intervention Usability Scale (IUS) - Goal Setting
The IUS is a 10-item measure with a possible total score ranging from 0 to 100; higher scores indicate a more usable intervention. Although this measure has not yet been normed as a measure of intervention usability, the System Usability Scale upon which the IUS is based defines scores of 70 or above as indicative of acceptable usability.
Time frame: This measure will be administered to participants in each training group after they complete the 25 hours of training.
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Traditional Training | Intervention Usability Scale (IUS) - Goal Setting | Pre-Training | 28.25 score on a scale | Standard Deviation 3.85 |
| Traditional Training | Intervention Usability Scale (IUS) - Goal Setting | Post-Training | 30.13 score on a scale | Standard Deviation 5.03 |
| ITS Based Training | Intervention Usability Scale (IUS) - Goal Setting | Pre-Training | 28.27 score on a scale | Standard Deviation 3.61 |
| ITS Based Training | Intervention Usability Scale (IUS) - Goal Setting | Post-Training | 30.82 score on a scale | Standard Deviation 4.02 |
Intervention Usability Scale (IUS) - Identifying Challenges
The IUS is a 10-item measure with a possible total score ranging from 0 to 100; higher scores indicate a more usable intervention. Although this measure has not yet been normed as a measure of intervention usability, the System Usability Scale upon which the IUS is based defines scores of 70 or above as indicative of acceptable usability.
Time frame: This measure will be administered to participants in each training group after they complete the 25 hours of training.
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Traditional Training | Intervention Usability Scale (IUS) - Identifying Challenges | Pre-Training | 26.75 score on a scale | Standard Deviation 4.3 |
| Traditional Training | Intervention Usability Scale (IUS) - Identifying Challenges | Post-Training | 28.88 score on a scale | Standard Deviation 4.09 |
| ITS Based Training | Intervention Usability Scale (IUS) - Identifying Challenges | Pre-Training | 27.91 score on a scale | Standard Deviation 4.55 |
| ITS Based Training | Intervention Usability Scale (IUS) - Identifying Challenges | Post-Training | 29.27 score on a scale | Standard Deviation 5.1 |