Mental Health Disorders
Conditions
Keywords
Quality Improvement, Organization of Services, Implementation
Brief summary
This randomized program evaluation is undertaken in conjunction with the Department of Veterans Affairs (VA) Office of Mental Health Operations (OMHO) and the Quality Enhancement Research Initiative. It is designed to answer two related questions: (1) Can an evidence-based implementation strategy using the Center for Disease Control (CDC)'s Replicating Effective Programs plus External Facilitation (REP-F) enhance the adoption of team-based care in VA General Mental Health (GMH) Clinics, and (2) Does the establishment of such teams via implementation enhance Veterans' health status, satisfaction, and perceptions of care? The model for team-based care is the evidence-based Collaborative Chronic Care Model (CCM). In conjunction with a nation-wide roll-out of the VA's Behavioral Health Interdisciplinary Program team (BHIP) initiative, the investigators have structured a randomized, controlled program evaluation to answer these questions. Specifically, using a stepped wedge design the investigators will randomize 9 VAMCs that have requested support in establishing a BHIP to 1 of 3 waves of REP-F support: immediate implementation support vs. 4-month vs. 8-month wait with dissemination of CCM materials (3 sites per wave). Fidelity and health outcome measures will be collected in a repeated measures design at 6-month intervals, and analyzed with general linear modeling.
Detailed description
Based on an internal system-wide review of mental health services and the Mental Health Action Plan submitted to Congress in November, 2011, the Office of Mental Health Operations (OMHO) has undertaken an effort to establish behavioral health interdisciplinary plans (BHIPs), which are intended to provide General Mental Health (GMH) care throughout the Department of Veterans Affairs (VA). The BHIP goal is to build effective interdisciplinary teams, which will provide the majority of care for Veterans in GMH. It is now expected that every Veterans Affairs Medical Center (VAMC) establish at least one BHIP in the current initial phase (begun in late fiscal year 2013), and that the effort scale-up subsequently. Not surprisingly, progress has been uneven. In 2015 OMHO incorporated the Collaborative Chronic Care Model (CCM) as an evidence-based model by which to structure BHIPs. Consistent with BHIP goals, CCMs were developed to provide anticipatory, continuous, collaborative, evidence-based care. CCMs consist of 6 elements: delivery system redesign, use of clinical information systems, provider decision support, patient self-management support, linkage to community resources, and healthcare organization support. Replicating Effective Programs with External Facilitation (REP-F) has been shown to be effective in implementing complex care models, including CCMs for mental health, both within and beyond VA. Thus in conjunction with OMHO, the investigators propose this project with the Specific Aim of evaluating the impact of REP-F in implementing CCM-based BHIPs and their effect on Veteran health status. The investigators propose a Hybrid Type III implementation-effectiveness stepped wedge controlled trial, specifically hypothesizing that: H1: REP-F-based implementation to establish CCM-based BHIPs, compared to existing centralized technical assistance will result in: (H1a) increased Veteran perceptions of CCM-based care, (H1b) higher rates of achieving national BHIP clinical fidelity measures (implementation outcomes), and (H1c) higher provider ratings of the presence of CCM elements. H2: CCM-based BHIPs, supported by REP-F implementation, will result in improved Veteran health outcomes compared to BHIPs supported by dissemination material alone (intervention outcomes). The investigators will utilize the national BHIP rollout as a vehicle for this project. Using a stepped wedge design the investigators will randomize 9 VAMCs that have requested support in establishing a BHIP to 1 of 3 waves of REP-F support: immediate implementation support vs. 4-month vs. 8-month wait with dissemination of CCM materials (3 sites per wave). Fidelity and health outcome measures will be collected in a repeated measures design at 6-month intervals, and analyzed with general linear modeling.
Interventions
OTHERReplicating Effective Programs plus External Facilitation
Packaging, training and technical assistance according to the Replicating Effective Programs model plus External Facilitation
OTHEREducational Materials
Dissemination of educational materials on the collaborative chronic care model for 4 or 8 months prior to cross-over to REP-F
Sponsors
US Department of Veterans Affairs
VA Boston Healthcare System
Study design
Allocation
RANDOMIZED
Intervention model
SEQUENTIAL
Primary purpose
TREATMENT
Masking
NONE
Eligibility
Sex/Gender
ALL
Age
18 Years to No maximum
Healthy volunteers
No
Inclusion criteria
At least three visits to the General Mental Health Clinic's BHIP team in prior year
Exclusion criteria
Chart evidence of dementia
Design outcomes
Primary
| Measure | Time frame | Description |
|---|---|---|
| Veterans RAND-12 Mental Component Score (VR-12 MCS) | one year | Veterans RAND-Mental Component Score: Overall self-rated mental health status over past two weeks among Veteran participants. Possible scores of minimum 0 and maximum 50. Higher is better. Administered with items for Veterans RAND-Physical Component Score. |
Secondary
| Measure | Time frame | Description |
|---|---|---|
| Satisfaction Index | One year | Satisfaction Index: Overall patient satisfaction with mental health services. Higher is better. Minimum score is 12, maximum score is 72. |
| Quality of Life Enjoyment and Satisfaction Questionnaire-Short Form (Q-LES-Q-SF) | One year | Recovery-oriented quality of life score. Higher is better. Minimum score 0, maximum score 100. |
| Patient Assessment of Chronic Illness Care (PACIC) | One year | Patient Assessment of Chronic Illness Care (PACIC): Veteran perception of coordination of and engagement with services. Higher is better. Minimum score is 1, maximum score is 33. |
| Team Development Measure (TDM)- Cohesion | baseline and during stepdown (6-12 months) | Mean percentage of participant provider team members who strongly agreed or agreed with the cohesion domain within the Team Development Measure. Measure Description: Team Development Measure (TDM) is a survey assessing team function at baseline and during the second six months of implementation using the four subscales with items rated from strongly agree to strongly disagree: Communication, Cohesion, Role Clarity, and Team Primacy (prioritizing team over individual goals). The percentage of team members agreeing or strongly agreeing with each statement was calculated, then averaged across all statements for a mean percentage for each subscale. Range of percentage was 0-100. |
| Veterans RAND-12 Physical Component Scores (VR-12 PCS) | One year | Veterans RAND-Physical Component Score: Overall self-rated physical health status over past two weeks among Veteran participants. Possible scores of minimum 0 and maximum 50. Higher is better. Administered with items for Veterans RAND-Mental Component Score. |
| Team Development Measure (TDM)- Role Clarity | baseline and during stepdown (6-12 months) | Mean percentage of participant team members who strongly agreed or agreed with the role clarity domain within the Team Development Measure. Measure Description: Team Development Measure (TDM) is a survey assessing team function at baseline and during the second six months of implementation using the four subscales with items rated from strongly agree to strongly disagree: Communication, Cohesion, Role Clarity, and Team Primacy (prioritizing team over individual goals). The percentage of team members agreeing or strongly agreeing with each statement was calculated, then averaged across all statements for a mean percentage for each subscale. Range of percentage was 0-100. Missing: cohesion (3); communication (2); role clarity (1); team primacy (0). |
| Team Development Measure (TDM)- Team Primacy | baseline and during stepdown (6-12 months) | Mean percentage of participant team members who strongly agreed or agreed with the team primacy domain within the Team Development Measure. Measure Description: Team Development Measure (TDM) is a survey assessing team function at baseline and during the second six months of implementation using the four subscales with items rated from strongly agree to strongly disagree: Communication, Cohesion, Role Clarity, and Team Primacy (prioritizing team over individual goals). The percentage of team members agreeing or strongly agreeing with each statement was calculated, then averaged across all statements for a mean percentage for each subscale. Range of percentage was 0-100. Missing: cohesion (3); communication (2); role clarity (1); team primacy (0). |
| Mental Health Hospitalization Rates | Two years | Quarterly indicator (0 not hospitalized or 1 hospitalized) for mental health hospitalization in the prior three months. |
| Team Development Measure (TDM)- Communication | baseline and during stepdown (6-12 months) | Mean percentage of participant team members who strongly agreed or agreed with the communication domain within the Team Development Measure. Measure Description: Measure Description: Team Development Measure (TDM) is a survey assessing team function at baseline and during the second six months of implementation using the four subscales with items rated from strongly agree to strongly disagree: Communication, Cohesion, Role Clarity, and Team Primacy (prioritizing team over individual goals). The percentage of team members agreeing or strongly agreeing with each statement was calculated, then averaged across all statements for a mean percentage for each subscale. Range of percentage was 0-100. |
Countries
United States
Participant flow
Participants by arm
| Arm | Count |
|---|---|
| Veteran Participants In this stepped wedge design, those eligible BHIP team-treated veterans randomly selected for telephone interview for health status measures. Primary analyses were within-subject. | 1,050 |
| Provider Participants In this stepped wedge design, BHIP team clinicians who participated in team function survey. Primary analyses were within-subject. | 62 |
| Total | 1,112 |
Baseline characteristics
| Characteristic | Veteran Participants | Provider Participants | Total |
|---|---|---|---|
| Age, Continuous | 53.9 years STANDARD_DEVIATION 13.7 | 49.6 years STANDARD_DEVIATION 9.9 | 52.405 years STANDARD_DEVIATION 14.4 |
| Depression or anxiety disorder | 592 Participants | — | 592 Participants |
| Employment Status (full-time, part-time, or self- employed) | 532 Participants | 62 Participants | 594 Participants |
| Marital Status (currently married) | 487 Participants | — | 487 Participants |
| Medical-surgical hospitalization in prior year | 73 Participants | — | 73 Participants |
| Mental health hospitalization in prior year | 36 Participants | — | 36 Participants |
| Number of active medical diagnoses (mean + sd) | 1.2 diagnoses STANDARD_DEVIATION 1.3 | — | 1.2 diagnoses STANDARD_DEVIATION 1.3 |
| Number of active mental health diagnoses (mean + sd) | 2.2 diagnoses STANDARD_DEVIATION 1.4 | — | 2.2 diagnoses STANDARD_DEVIATION 1.4 |
| Period of service (Gulf War or later, 1990-present) | 497 Participants | — | 497 Participants |
| Post traumatic stress disorder | 474 Participants | — | 474 Participants |
| Race/Ethnicity, Customized Black | 159 Participants | — | 159 Participants |
| Race/Ethnicity, Customized Hispanic or Latino | 97 Participants | — | 97 Participants |
| Race/Ethnicity, Customized Minority | 275 Participants | — | 275 Participants |
| Race/Ethnicity, Customized Other | 26 Participants | — | 26 Participants |
| Race/Ethnicity, Customized White | 825 Participants | — | 825 Participants |
| Rural residence (rural and highly rural) | 227 Participants | — | 227 Participants |
| Serious Mental Illness (bipolar spectrum or schizophrenia) | 340 Participants | — | 340 Participants |
| Sex: Female, Male Female | 210 Participants | 35 Participants | 245 Participants |
| Sex: Female, Male Male | 840 Participants | 27 Participants | 867 Participants |
| Substance use disorder | 169 Participants | — | 169 Participants |
| Team Development Measure (TDM) Cohesion | — | 84.0 scores on a scale STANDARD_DEVIATION 20.8 | 84.0 scores on a scale STANDARD_DEVIATION 20.8 |
| Team Development Measure (TDM) Communication | — | 83.3 scores on a scale STANDARD_DEVIATION 24.6 | 83.3 scores on a scale STANDARD_DEVIATION 24.6 |
| Team Development Measure (TDM) Role Clarity | — | 53.4 scores on a scale STANDARD_DEVIATION 26.9 | 53.4 scores on a scale STANDARD_DEVIATION 26.9 |
| Team Development Measure (TDM) Team Primacy | — | 50.0 scores on a scale STANDARD_DEVIATION 37.6 | 50.0 scores on a scale STANDARD_DEVIATION 37.6 |
| VA disability % (>= 50%) | 844 Participants | — | 844 Participants |
Adverse events
| Event type | EG000 affected / at risk | EG001 affected / at risk |
|---|---|---|
| deaths Total, all-cause mortality | 0 / 1,050 | 0 / 62 |
| other Total, other adverse events | 0 / 1,050 | 0 / 62 |
| serious Total, serious adverse events | 0 / 1,050 | 0 / 62 |
Outcome results
Primary
Veterans RAND-12 Mental Component Score (VR-12 MCS)
Veterans RAND-Mental Component Score: Overall self-rated mental health status over past two weeks among Veteran participants. Possible scores of minimum 0 and maximum 50. Higher is better. Administered with items for Veterans RAND-Physical Component Score.
Time frame: one year
Population: Adjusted mixed model of single group with repeated event. Number Analyzed = number of participants with available data at the specified time points. 1050 participants completed 1436 interviews across one or more time points: t0, t6, t12. 589 started at t0 (missing=4); 65 added at t6 (missing=1); 386 added at t12 (missing=5).
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Veteran Participants | Veterans RAND-12 Mental Component Score (VR-12 MCS) | Baseline (T0) | 36.82 score on a scale | Standard Deviation 14.78 |
| Veteran Participants | Veterans RAND-12 Mental Component Score (VR-12 MCS) | Six Month Timepoint (T6) | 36.24 score on a scale | Standard Deviation 14.26 |
| Veteran Participants | Veterans RAND-12 Mental Component Score (VR-12 MCS) | Final Twelve Month Timepoint (T12) | 37.25 score on a scale | Standard Deviation 14.07 |
p-value: <0.0195% CI: [-1.26, 1.5]Mixed Models Analysis
Comparison: Among Veteran participants,we used a linear contrast comparing a) those with complex clinical presentations, defined as receiving treatment for three or more mental health diagnoses in the prior year to b) those with two or fewer diagnoses during the facilitation year from T0 to T12..p-value: <0.00195% CI: [2.24, 7.82]Regression, Linear
Secondary
Mental Health Hospitalization Rates
Quarterly indicator (0 not hospitalized or 1 hospitalized) for mental health hospitalization in the prior three months.
Time frame: Two years
Population: All eligible patients were coded for hospitalization yes/no in each quarter.
| Arm | Measure | Value (LEAST_SQUARES_MEAN) |
|---|---|---|
| Veteran Participants | Mental Health Hospitalization Rates | -0.12 beta coefficient |
p-value: <0.00195% CI: [-0.16, -0.07]Mixed Models Analysis
Secondary
Patient Assessment of Chronic Illness Care (PACIC)
Patient Assessment of Chronic Illness Care (PACIC): Veteran perception of coordination of and engagement with services. Higher is better. Minimum score is 1, maximum score is 33.
Time frame: One year
Population: Adjusted mixed model of a single group with repeated event. Number Analyzed = number of participants with available data and the specified time points. 1050 completed 1436 interviews across t0, t6, t12. 582 started at t0 (missing=11); 65 added at t6 (missing=1); 384 added at t12 (missing=7). Site included as random effect.
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Veteran Participants | Patient Assessment of Chronic Illness Care (PACIC) | T0 | 21.94 score on a scale | Standard Deviation 6.35 |
| Veteran Participants | Patient Assessment of Chronic Illness Care (PACIC) | T6 | 22.46 score on a scale | Standard Deviation 6.57 |
| Veteran Participants | Patient Assessment of Chronic Illness Care (PACIC) | T12 | 21.99 score on a scale | Standard Deviation 6.4 |
p-value: >0.0595% CI: [-0.6, 0.8]Mixed Models Analysis
Secondary
Quality of Life Enjoyment and Satisfaction Questionnaire-Short Form (Q-LES-Q-SF)
Recovery-oriented quality of life score. Higher is better. Minimum score 0, maximum score 100.
Time frame: One year
Population: Adjusted mixed model of a single group with repeated event. Number Analyzed = number of participants with available data and the specified time points. 1050 completed 1436 interviews across t0, t6, t12. 577 started at t0 (missing=16), 63 added at t6 (missing=3); 380 added at t12 (missing=11). Site included as random effect.
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Veteran Participants | Quality of Life Enjoyment and Satisfaction Questionnaire-Short Form (Q-LES-Q-SF) | Baseline (T0) | 53.2 score on a scale | Standard Deviation 19.33 |
| Veteran Participants | Quality of Life Enjoyment and Satisfaction Questionnaire-Short Form (Q-LES-Q-SF) | Six Month Timepoint (T6) | 53.9 score on a scale | Standard Deviation 20.82 |
| Veteran Participants | Quality of Life Enjoyment and Satisfaction Questionnaire-Short Form (Q-LES-Q-SF) | Final Twelve Month Timepoint (T12) | 54.8 score on a scale | Standard Deviation 19.65 |
p-value: >0.0595% CI: [-1.3, 2.3]Mixed Models Analysis
Secondary
Satisfaction Index
Satisfaction Index: Overall patient satisfaction with mental health services. Higher is better. Minimum score is 12, maximum score is 72.
Time frame: One year
Population: Adjusted mixed model of a single group with repeated event. Number Analyzed = number of participants with available data and the specified time points. 1050 completed 1436 interviews across t0, t6, t12. 576 started at t0 (missing=17); 64 added at t6 (missing=2); 380 added at t12 (missing=11). Site included as random effect.
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Veteran Participants | Satisfaction Index | T0 | 52.94 score on a scale | Standard Deviation 14.4 |
| Veteran Participants | Satisfaction Index | T6 | 53.26 score on a scale | Standard Deviation 15.08 |
| Veteran Participants | Satisfaction Index | T12 | 52.78 score on a scale | Standard Deviation 15.18 |
p-value: >0.0595% CI: [-0.2, 0.9]Mixed Models Analysis
Secondary
Team Development Measure (TDM)- Cohesion
Mean percentage of participant provider team members who strongly agreed or agreed with the cohesion domain within the Team Development Measure. Measure Description: Team Development Measure (TDM) is a survey assessing team function at baseline and during the second six months of implementation using the four subscales with items rated from strongly agree to strongly disagree: Communication, Cohesion, Role Clarity, and Team Primacy (prioritizing team over individual goals). The percentage of team members agreeing or strongly agreeing with each statement was calculated, then averaged across all statements for a mean percentage for each subscale. Range of percentage was 0-100.
Time frame: baseline and during stepdown (6-12 months)
Population: Total 62 mental health providers who completed survey. Number Analyzed = number of participants with available data at the specified time points for Cohesion domain (missing=3). 40 completed at 0 months (t0) and 43 completed at 12 months (t12). 21 new providers completed at t12; 22 completed at both t0 and t12. 19 completed only t0.
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Veteran Participants | Team Development Measure (TDM)- Cohesion | Baseline (T0) | 84.0 score on a scale | Standard Deviation 20.8 |
| Veteran Participants | Team Development Measure (TDM)- Cohesion | Follow-up/Final at Six to Twelve Months (T12) | 84.5 score on a scale | Standard Deviation 22.4 |
p-value: >0.0595% CI: [-0.074, 0.084]t-test, 2 sided
Secondary
Team Development Measure (TDM)- Communication
Mean percentage of participant team members who strongly agreed or agreed with the communication domain within the Team Development Measure. Measure Description: Measure Description: Team Development Measure (TDM) is a survey assessing team function at baseline and during the second six months of implementation using the four subscales with items rated from strongly agree to strongly disagree: Communication, Cohesion, Role Clarity, and Team Primacy (prioritizing team over individual goals). The percentage of team members agreeing or strongly agreeing with each statement was calculated, then averaged across all statements for a mean percentage for each subscale. Range of percentage was 0-100.
Time frame: baseline and during stepdown (6-12 months)
Population: Total 62 mental health providers who completed survey. Number Analyzed = number of participants with available data and the specified time points for Communication domain (missing=2). 40 completed at 0 months (t0) and 43 completed at 12 months (t12). 21 new providers completed at t12; 22 completed at both t0 and t12. 19 completed only t0.
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Veteran Participants | Team Development Measure (TDM)- Communication | Baseline (T0) | 83.3 score on a scale | Standard Deviation 24.6 |
| Veteran Participants | Team Development Measure (TDM)- Communication | Follow-up/Final at Six to Twelve Months (T12) | 84.4 score on a scale | Standard Deviation 20.6 |
p-value: >0.0595% CI: [-0.056, 0.077]t-test, 2 sided
Secondary
Team Development Measure (TDM)- Role Clarity
Mean percentage of participant team members who strongly agreed or agreed with the role clarity domain within the Team Development Measure. Measure Description: Team Development Measure (TDM) is a survey assessing team function at baseline and during the second six months of implementation using the four subscales with items rated from strongly agree to strongly disagree: Communication, Cohesion, Role Clarity, and Team Primacy (prioritizing team over individual goals). The percentage of team members agreeing or strongly agreeing with each statement was calculated, then averaged across all statements for a mean percentage for each subscale. Range of percentage was 0-100. Missing: cohesion (3); communication (2); role clarity (1); team primacy (0).
Time frame: baseline and during stepdown (6-12 months)
Population: Total 62 mental health providers who completed survey.Number Analyzed = number of participants with available data and the specified time points for Role Clarity domain (missing= 1). 40 completed at 0 months (t0) and 43 completed at 12 months (t12). 21 new providers completed at t12; 22 completed at both t0 and t12. 19 completed only t0.
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Veteran Participants | Team Development Measure (TDM)- Role Clarity | Baseline (T0) | 53.4 score on a scale | Standard Deviation 26.9 |
| Veteran Participants | Team Development Measure (TDM)- Role Clarity | Follow-up/Final at Six to Twelve Months (T12) | 68.7 score on a scale | Standard Deviation 28.7 |
p-value: <0.00195% CI: [0.044, 0.262]t-test, 2 sided
Secondary
Team Development Measure (TDM)- Team Primacy
Mean percentage of participant team members who strongly agreed or agreed with the team primacy domain within the Team Development Measure. Measure Description: Team Development Measure (TDM) is a survey assessing team function at baseline and during the second six months of implementation using the four subscales with items rated from strongly agree to strongly disagree: Communication, Cohesion, Role Clarity, and Team Primacy (prioritizing team over individual goals). The percentage of team members agreeing or strongly agreeing with each statement was calculated, then averaged across all statements for a mean percentage for each subscale. Range of percentage was 0-100. Missing: cohesion (3); communication (2); role clarity (1); team primacy (0).
Time frame: baseline and during stepdown (6-12 months)
Population: Total 62 mental health providers who completed survey. Number Analyzed = number of participants with available data and the specified time points for Team Primacy (0 missing). 40 completed at 0 months (t0) and 43 completed at 12 months (t12). 21 new providers completed at t12; 22 completed at both t0 and t12. 19 completed only t0.
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Veteran Participants | Team Development Measure (TDM)- Team Primacy | Baseline (T0) | 50.0 score on a scale | Standard Deviation 37.6 |
| Veteran Participants | Team Development Measure (TDM)- Team Primacy | Follow-up/Final at Six to Twelve Months (T12) | 68.6 score on a scale | Standard Deviation 28.7 |
p-value: <0.0195% CI: [0.083, 0.289]t-test, 2 sided
Secondary
Veterans RAND-12 Physical Component Scores (VR-12 PCS)
Veterans RAND-Physical Component Score: Overall self-rated physical health status over past two weeks among Veteran participants. Possible scores of minimum 0 and maximum 50. Higher is better. Administered with items for Veterans RAND-Mental Component Score.
Time frame: One year
Population: Adjusted mixed model of a single group with repeated event. Number Analyzed = number of participants with available data and the specified time points. 1050 completed 1436 interviews across t0, t6, t12. 593 started at t0; 66 added at t6; 391 added at t12. None missing from survey. Site included as random effect.
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Veteran Participants | Veterans RAND-12 Physical Component Scores (VR-12 PCS) | T0 | 35.20 score on a scale | Standard Deviation 13.11 |
| Veteran Participants | Veterans RAND-12 Physical Component Scores (VR-12 PCS) | T6 | 35.62 score on a scale | Standard Deviation 12.61 |
| Veteran Participants | Veterans RAND-12 Physical Component Scores (VR-12 PCS) | T12 | 35.30 score on a scale | Standard Deviation 12.31 |
p-value: <0.0495% CI: [0.04, 2.3]Mixed Models Analysis