Bipolar Disorder, Depressive Disorder, Schizoaffective Disorder, Schizophrenia
Conditions
Brief summary
This is a randomized controlled trial to evaluate the community-based effectiveness of virtual reality job interview training (VR-JIT). Northwestern University is partnering with Thresholds Inc. to evaluate the effectiveness of VR-JIT at improving interviewing skills and access to employment. In addition, we will evaluate the cost effectiveness of VR-JIT and the process for implementing VR-JIT at Thresholds.
Interventions
BEHAVIORALVirtual Reality Job Interview Training
Job interview training using virtual reality software created by SIMmersion, Inc, in addition to standard supported employment services.
OTHERSupported Employment (Services as Usual)
Standard supported employment services.
Sponsors
National Institute of Mental Health (NIMH)
Thresholds Inc.
SIMmersion, LLC
University of Utah
University of Michigan
Study design
Allocation
RANDOMIZED
Intervention model
PARALLEL
Primary purpose
OTHER
Masking
SINGLE (Outcomes Assessor)
Masking description
Some, but not all, outcomes require masking for the assessor.
Eligibility
Sex/Gender
ALL
Age
18 Years to No maximum
Healthy volunteers
No
Inclusion criteria
1. Over the age of 18 2. Diagnosis of any of the following: 1. Any chronic schizophrenia spectrum disorder, including schizophrenia, schizoaffective disorder, delusional disorder, brief psychotic disorder, schizophreniform disorder, other specified schizophrenia spectrum and related disorder, and unspecified schizophrenia spectrum and related disorders; 2. Any bipolar spectrum disorder beyond bipolar type I and II to include cyclothymia, other specified bipolar and related disorder, and unspecified bipolar and related disorders; 3. Any depressive disorder, including disruptive mood dysregulation disorder, major depressive disorder, persistent depressive disorder, other specified depressive disorder, and unspecified depressive disorder; 4. Posttraumatic stress disorder 3. fluency in English 4. 6th grade reading level 5. Enrolled in Supported Employment/Individual Placement Support (IPS) 6. Active IPS enrollee as indicated by at least 1 contact with his/her Employment Specialist in the past 30 days. 7. Currently unemployed or underemployed 8. Planning to go on interviews within 4 weeks of registration on study. 9. Willing to be video recorded for certain study elements. 10. Must have given signed, informed consent prior to registration on study.
Exclusion criteria
1. Documented developmental or learning disability. 2. Medical condition that may compromise cognition (e.g., Parkinson's Disease, Alzheimer's Disease, Huntington's Chorea, Moderate or Greater TBI). 3. Uncorrected vision or hearing problems 4. Active suicidal ideation within the last 30 days with at least some intent to act, with or without a specific plan; this would be reflected in a score of 4 or 5 on the Suicidal Ideation section of the Columbia Suicide Severity Rating Scale (CSSRS) 5. A past suicide attempt within the past 30 days that did not include preparatory acts or behavior, but was defined by a potentially self-injurious act committed with at least some wish to die as a result of the act.
Design outcomes
Primary
| Measure | Time frame | Description |
|---|---|---|
| Employment Rates | 9 month Follow-Up from Randomization Date | Employment rates at 9-month follow-up visit measured with questionnaire. |
| Time to Employment | 9 month Follow-Up from Randomization Date | We evaluated the number of days between randomization and obtaining employment using a questionnaire administered over phone interviews with the subject and also verifying with Supported Employment (IPS) tracking system. |
Secondary
| Measure | Time frame | Description |
|---|---|---|
| Interview Anxiety | Pretest (Baseline) and Posttest (approximately 3 months after Baseline) | Self-reported measure of interviewing anxiety completed at pretest and posttest. Participants rated their interview anxiety with an adapted version of the 34-item Personal Report of Public Speaking Apprehension (PRSPA); the phrase publics peaking was replaced with job interviewing. Items were coded from 1, strongly disagree, to 5, strongly agree. Total scores were computed by using the PRSPA's validated two-step total score procedure with the minimum score being 34 and maximum score being 170. Higher scores indicate higher interview anxiety. |
| Job Interview Skills | Pretest (Baseline) and Posttest (approximately 3 months after Baseline) | Interviewing skills will be measured using a role-play measure in which subjects act out two job interview scenarios with trained actors. Interviews will be video recorded and scored. An average of the two scenarios at baseline was used to create a single pretest score and an average of the two scenarios at posttest was used to create a single posttest score. The method of assessment is the job interview skills rating scale as measured by the Mock Interview Rating Scale. Possible scores range from 8 to 40, with higher scores indicating better interview skills. |
| Psychological Distress - Brief Psychiatric Rating Scale | Pretest (Baseline) and 6 Month Follow Up from Randomization Date | The Brief Psychiatric Rating Scale (BPRS) consists of 18 items and uses a seven-item Likert scale with 1 = not present, and 7 = extremely severe. Sum scores can range from 18 to 126. Higher scores indicate more symptoms. |
| Social Competence | Pretest (Baseline) and Posttest (approximately 3 months after Baseline) | Measured by Social Skills Performance Assessment (SSPA). Roleplay performance: The SSPA includes two brief (3-minute) roleplays in which participants engage in a conversation with an unknown confederate who plays the role of a new neighbor(NN) or a landlord (LL). The NN and LL role plays were video recorded at the pre- and posttest visits and were blindly rated by use of an anchoring system. The NN roleplay was scored on eight items via a 5-point scale (e.g., 5,very interested, to 1, very disinterested), and the LL roleplay was scored on nine items via a 5-point scale (e.g., 5,very focused, to 1, very unfocused). The two role-play scores were combined to create one total pretest score and one total posttest score. Sum scores could have a minimum of 17 and maximum of 85. Higher scores indicated better skills. |
| Interview Self-Confidence | Pretest (Baseline) and Posttest (approximately 3 months after Baseline) | Self-reported interview self-confidence. Participants self-reported their confidence in interview skills at pretest and posttest by using a nine-item survey, with items rated on a 7-point scale (1, extremely unskilled, to 7, extremely skilled); total scores were computed across all items. The minimum score is 9 and the maximum score would be 63. Higher scores indicate higher self-confidence in interview skills. |
Countries
United States
Participant flow
Pre-assignment details
Randomized Control Trial Phase: 105 people were enrolled and signed consent forms, but 6 people dropped out of the study before randomization and were not assigned to a group. Therefore, the Randomized Control Trial Phase includes 99 people who signed consent forms and were actually randomized into the study. Focus Group Phase: This includes both Thresholds staff focus groups as well as focus groups completed by those who received the intervention in the Randomized Control Trial Phase.
Participants by arm
| Arm | Count |
|---|---|
| Supported Employment (Services as Usual) Supported Employment (Services as Usual)
Supported Employment (Services as Usual): Standard supported employment services. | 38 |
| Virtual Reality Job Interview Training Virtual Reality Job Interview Training
Virtual Reality Job Interview Training: Job interview training using virtual reality software created by SIMmersion, Inc, in addition to standard supported employment services. | 61 |
| Thresholds Staff Only sex, ethnicity, and race were collected for the Thresholds Staff group.
Not other baseline data was collected for Thresholds Staff. | 44 |
| Focus Groups Baseline data were not collected for the Focus Groups. | 25 |
| Total | 168 |
Withdrawals & dropouts
| Period | Reason | FG000 | FG001 | FG002 | FG003 |
|---|---|---|---|---|---|
| Randomized Control Trial Phase | COVID Quarantine | 2 | 3 | 0 | 0 |
| Randomized Control Trial Phase | Lost to Follow-up | 0 | 2 | 3 | 0 |
| Randomized Control Trial Phase | Participant placed in nursing home | 0 | 1 | 0 | 0 |
| Randomized Control Trial Phase | Participant was in another concurrent study | 0 | 1 | 0 | 0 |
Baseline characteristics
| Characteristic | Virtual Reality Job Interview Training | Supported Employment (Services as Usual) | Total | Thresholds Staff | Focus Groups |
|---|---|---|---|---|---|
| Age, Continuous | 47.4 years STANDARD_DEVIATION 12.7 | 44.5 years STANDARD_DEVIATION 12.8 | 46.3 years STANDARD_DEVIATION 12.8 | — | — |
| Brief Psychiatric Rating Scale | 24.0 units on a scale STANDARD_DEVIATION 5.7 | 26.4 units on a scale STANDARD_DEVIATION 7.2 | 24.9 units on a scale STANDARD_DEVIATION 6.4 | — | — |
| Cognitive ability Neurocognition | 34.6 T-score STANDARD_DEVIATION 8.3 | 34.2 T-score STANDARD_DEVIATION 10 | 34.4 T-score STANDARD_DEVIATION 8.9 | — | — |
| Cognitive ability Social cognition | 37.4 T-score STANDARD_DEVIATION 16.8 | 35.2 T-score STANDARD_DEVIATION 14 | 36.5 T-score STANDARD_DEVIATION 15.7 | — | — |
| Community Functioning | 130.6 units on a scale STANDARD_DEVIATION 12.2 | 127.3 units on a scale STANDARD_DEVIATION 14.6 | 129.4 units on a scale STANDARD_DEVIATION 13.2 | — | — |
| Days in IPS at Baseline | 118.2 days STANDARD_DEVIATION 132.6 | 167.9 days STANDARD_DEVIATION 189.4 | 138.1 days STANDARD_DEVIATION 158.7 | — | — |
| Employment History: Ever employed full-time | 44 Participants | 25 Participants | 69 Participants | 0 Participants | — |
| Employment History: Ever employed part-time | 50 Participants | 30 Participants | 80 Participants | — | — |
| Employment History: Had competitive employment in past 2 years | 24 Participants | 20 Participants | 44 Participants | — | — |
| Ethnicity (NIH/OMB) Hispanic or Latino | 7 Participants | 2 Participants | 13 Participants | 4 Participants | — |
| Ethnicity (NIH/OMB) Not Hispanic or Latino | 54 Participants | 36 Participants | 126 Participants | 36 Participants | — |
| Ethnicity (NIH/OMB) Unknown or Not Reported | 0 Participants | 0 Participants | 4 Participants | 4 Participants | — |
| Primary Psychiatric Diagnosis Bipolar Disorder | 15 Participants | 12 Participants | 27 Participants | — | — |
| Primary Psychiatric Diagnosis Major Depressive Disorder | 23 Participants | 8 Participants | 31 Participants | — | — |
| Primary Psychiatric Diagnosis Other | 0 Participants | 1 Participants | 1 Participants | — | — |
| Primary Psychiatric Diagnosis Schizoaffective Disorder | 9 Participants | 8 Participants | 17 Participants | — | — |
| Primary Psychiatric Diagnosis Schizophrenia | 14 Participants | 8 Participants | 22 Participants | — | — |
| Primary Psychiatric Diagnosis Unknown or Not Reported | 0 Participants | 1 Participants | 1 Participants | — | — |
| Race/Ethnicity, Customized Black, African American | 37 Participants | 26 Participants | 82 Participants | 19 Participants | — |
| Race/Ethnicity, Customized More than one race | 2 Participants | 0 Participants | 3 Participants | 1 Participants | — |
| Race/Ethnicity, Customized Other | 1 Participants | 0 Participants | 2 Participants | 1 Participants | — |
| Race/Ethnicity, Customized White | 21 Participants | 12 Participants | 52 Participants | 19 Participants | — |
| Region of Enrollment United States | 61 participants | 38 participants | 25 participants | 44 participants | 25 participants |
| Sex: Female, Male Female | 32 Participants | 15 Participants | 73 Participants | 26 Participants | 0 Participants |
| Sex: Female, Male Male | 29 Participants | 23 Participants | 68 Participants | 16 Participants | 0 Participants |
| Substance Use Alcohol Use Disorders Identification Test (AUDIT) | 3.1 units on a scale STANDARD_DEVIATION 4.7 | 5.3 units on a scale STANDARD_DEVIATION 7.3 | 4.0 units on a scale STANDARD_DEVIATION 5.9 | — | — |
| Substance Use Drug Abuse Screening Test (DAST) | 1.4 units on a scale STANDARD_DEVIATION 1 | 1.6 units on a scale STANDARD_DEVIATION 1.2 | 1.5 units on a scale STANDARD_DEVIATION 1.1 | — | — |
| Years of Education | 12.6 years STANDARD_DEVIATION 2.2 | 12.7 years STANDARD_DEVIATION 1.8 | 12.7 years STANDARD_DEVIATION 2.1 | — | — |
Adverse events
| Event type | EG000 affected / at risk | EG001 affected / at risk |
|---|---|---|
| deaths Total, all-cause mortality | 0 / 38 | 0 / 59 |
| other Total, other adverse events | 0 / 38 | 0 / 59 |
| serious Total, serious adverse events | 0 / 38 | 0 / 59 |
Outcome results
Primary
Employment Rates
Employment rates at 9-month follow-up visit measured with questionnaire.
Time frame: 9 month Follow-Up from Randomization Date
| Arm | Measure | Value (COUNT_OF_PARTICIPANTS) |
|---|---|---|
| Virtual Reality Job Interview Training | Employment Rates | 23 Participants |
| Supported Employment (Services as Usual) | Employment Rates | 10 Participants |
p-value: 0.027Regression, Logistic
p-value: 0.0765Chi-squared
Primary
Time to Employment
We evaluated the number of days between randomization and obtaining employment using a questionnaire administered over phone interviews with the subject and also verifying with Supported Employment (IPS) tracking system.
Time frame: 9 month Follow-Up from Randomization Date
Population: Time to employment was only reported for those participants who provided that data and who were employed. Not everyone was employed so they would not have a date or a time to employment within the analysis. Therefore, this analysis only reports on the 36 people who did get jobs.
| Arm | Measure | Value (MEAN) | Dispersion |
|---|---|---|---|
| Virtual Reality Job Interview Training | Time to Employment | 60.6 days | Standard Deviation 57.1 |
| Supported Employment (Services as Usual) | Time to Employment | 101.3 days | Standard Deviation 76.1 |
p-value: 0.062Regression, Cox
Secondary
Interview Anxiety
Self-reported measure of interviewing anxiety completed at pretest and posttest. Participants rated their interview anxiety with an adapted version of the 34-item Personal Report of Public Speaking Apprehension (PRSPA); the phrase publics peaking was replaced with job interviewing. Items were coded from 1, strongly disagree, to 5, strongly agree. Total scores were computed by using the PRSPA's validated two-step total score procedure with the minimum score being 34 and maximum score being 170. Higher scores indicate higher interview anxiety.
Time frame: Pretest (Baseline) and Posttest (approximately 3 months after Baseline)
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Virtual Reality Job Interview Training | Interview Anxiety | Pretest | 100.2 score on a scale | Standard Error 3.6 |
| Virtual Reality Job Interview Training | Interview Anxiety | Posttest | 90.4 score on a scale | Standard Error 3.8 |
| Supported Employment (Services as Usual) | Interview Anxiety | Pretest | 90.5 score on a scale | Standard Error 4.5 |
| Supported Employment (Services as Usual) | Interview Anxiety | Posttest | 88.9 score on a scale | Standard Error 4.6 |
p-value: 0.019Regression, Linear
Secondary
Interview Self-Confidence
Self-reported interview self-confidence. Participants self-reported their confidence in interview skills at pretest and posttest by using a nine-item survey, with items rated on a 7-point scale (1, extremely unskilled, to 7, extremely skilled); total scores were computed across all items. The minimum score is 9 and the maximum score would be 63. Higher scores indicate higher self-confidence in interview skills.
Time frame: Pretest (Baseline) and Posttest (approximately 3 months after Baseline)
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Virtual Reality Job Interview Training | Interview Self-Confidence | Pretest | 43 score on a scale | Standard Error 1.5 |
| Virtual Reality Job Interview Training | Interview Self-Confidence | Posttest | 47.6 score on a scale | Standard Error 1.6 |
| Supported Employment (Services as Usual) | Interview Self-Confidence | Pretest | 43.7 score on a scale | Standard Error 1.7 |
| Supported Employment (Services as Usual) | Interview Self-Confidence | Posttest | 43.8 score on a scale | Standard Error 1.8 |
p-value: 0.013Regression, Linear
Secondary
Job Interview Skills
Interviewing skills will be measured using a role-play measure in which subjects act out two job interview scenarios with trained actors. Interviews will be video recorded and scored. An average of the two scenarios at baseline was used to create a single pretest score and an average of the two scenarios at posttest was used to create a single posttest score. The method of assessment is the job interview skills rating scale as measured by the Mock Interview Rating Scale. Possible scores range from 8 to 40, with higher scores indicating better interview skills.
Time frame: Pretest (Baseline) and Posttest (approximately 3 months after Baseline)
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Virtual Reality Job Interview Training | Job Interview Skills | Pretest | 24.6 score on a scale | Standard Error 0.8 |
| Virtual Reality Job Interview Training | Job Interview Skills | Posttest | 24.8 score on a scale | Standard Error 0.8 |
| Supported Employment (Services as Usual) | Job Interview Skills | Pretest | 25.6 score on a scale | Standard Error 0.6 |
| Supported Employment (Services as Usual) | Job Interview Skills | Posttest | 27.4 score on a scale | Standard Error 0.6 |
p-value: 0.006Regression, Linear
Secondary
Psychological Distress - Brief Psychiatric Rating Scale
The Brief Psychiatric Rating Scale (BPRS) consists of 18 items and uses a seven-item Likert scale with 1 = not present, and 7 = extremely severe. Sum scores can range from 18 to 126. Higher scores indicate more symptoms.
Time frame: Pretest (Baseline) and 6 Month Follow Up from Randomization Date
Population: For the outcome measure table, 99 participants (38 Supported Employment and 61 Virtual Reality Job Interview Training) completed the pretest Brief Psychiatric (BPRS) at baseline. The total number dropped to 58 participants (25 Supported Employment and 33 Virtual Reality Job Interview Training) at posttest due to participant drop out or non completion of the 6 Month Follow Up visit.
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Virtual Reality Job Interview Training | Psychological Distress - Brief Psychiatric Rating Scale | Pretest BPRS Total | 26.4 score on a scale | Standard Deviation 7.2 |
| Virtual Reality Job Interview Training | Psychological Distress - Brief Psychiatric Rating Scale | 6 Month BPRS Total | 24.6 score on a scale | Standard Deviation 9.4 |
| Supported Employment (Services as Usual) | Psychological Distress - Brief Psychiatric Rating Scale | Pretest BPRS Total | 24.0 score on a scale | Standard Deviation 5.7 |
| Supported Employment (Services as Usual) | Psychological Distress - Brief Psychiatric Rating Scale | 6 Month BPRS Total | 23.3 score on a scale | Standard Deviation 9.2 |
Secondary
Social Competence
Measured by Social Skills Performance Assessment (SSPA). Roleplay performance: The SSPA includes two brief (3-minute) roleplays in which participants engage in a conversation with an unknown confederate who plays the role of a new neighbor(NN) or a landlord (LL). The NN and LL role plays were video recorded at the pre- and posttest visits and were blindly rated by use of an anchoring system. The NN roleplay was scored on eight items via a 5-point scale (e.g., 5,very interested, to 1, very disinterested), and the LL roleplay was scored on nine items via a 5-point scale (e.g., 5,very focused, to 1, very unfocused). The two role-play scores were combined to create one total pretest score and one total posttest score. Sum scores could have a minimum of 17 and maximum of 85. Higher scores indicated better skills.
Time frame: Pretest (Baseline) and Posttest (approximately 3 months after Baseline)
Population: 96 participants (37 Supported Employment and 59 Virtual Reality Job Interview Training) completed the pretest Social Skills Performance Assessment (SSPA) at baseline. We are missing data on 3 participants due to technical difficulties (no video recording to score) or participants refusing to complete. The total number dropped to 67 participants (27 Supported Employment and 40 Virtual Reality Job Interview Training) at posttest due to participant drop out or non completion of the posttest visit.
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Virtual Reality Job Interview Training | Social Competence | Pretest | 3.8 score on a scale | Standard Deviation 0.6 |
| Virtual Reality Job Interview Training | Social Competence | Posttest | 4.0 score on a scale | Standard Deviation 0.6 |
| Supported Employment (Services as Usual) | Social Competence | Pretest | 3.7 score on a scale | Standard Deviation 0.7 |
| Supported Employment (Services as Usual) | Social Competence | Posttest | 3.8 score on a scale | Standard Deviation 0.7 |
Comparison: For this analysis, only participants who completed both pre-test and posttest were included. If someone completed a pre-test SSPA, but not a posttest SSPA, they were not included in the analysis.p-value: 0.692Regression, Linear