Organizational Support for Evidence-based Public Health
Conditions
Keywords
Dissemination research, Public health workforce, Evidence-based public health, Public health accreditation
Brief summary
The purpose of this study is to identify and evaluate dissemination strategies to support the uptake of evidence-based programs and policies (EBPPs) for diabetes prevention and control among local-level public health practitioners. Dissemination strategies such as multi-day in-person training workshops, electronic information exchange modalities, and remote technical assistance are hypothesized to associate with improved access and use of public health evidence and organizational supports for program and policy decision making based on evidence-based public health.
Detailed description
Evidence-based public health approaches to prevent and control diabetes and other chronic diseases have been identified in recent decades, and could have a profound effect on diabetes incidence and quality and length of life of those diagnosed. However, barriers to implement approaches continue because of lack of organizational support, limited resources, competing priorities, and limited skill among the public health workforce. The purpose of this study is to determine effective ways to promote the adoption of evidence based public health practice related to diabetes and chronic disease prevention and control among local health departments (LHDs). This stepped-wedge cluster randomized trial aims to evaluate active dissemination strategies on local-level public health practitioners to increase adoption and use of evidence-based programs and policies for diabetes and chronic disease prevention and control among LHDs in Missouri. Twelve LHDs will be recruited and randomly assigned to one of three groups that cross over from usual practice to receive the intervention (dissemination) strategies at 8-month intervals; the intervention duration for groups ranges from 8 to 24 months. LHD staff and the university-based study team are jointly identifying, refining and selecting dissemination strategies. Intervention strategies may include multi-day in-person training workshops, electronic information exchange modalities, and remote technical assistance. Evaluation methods include surveys at baseline and at each 8-month interval, abstraction of LHD chronic disease prevention program plans and progress reports, and social network analysis.
Interventions
Participating local health departments will help develop and choose dissemination strategies they prefer for their staff working in and supporting diabetes and chronic disease prevention and control to receive. Dissemination strategies may include multi-day in-person training workshops, electronic information exchange modalities, remote technical assistance, and information on ways to enhance organizational climates favorable to evidence-based diabetes prevention and control.
Sponsors
National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK)
Washington University School of Medicine
Study design
Allocation
RANDOMIZED
Intervention model
CROSSOVER
Primary purpose
OTHER
Masking
NONE
Eligibility
Sex/Gender
ALL
Age
21 Years to No maximum
Healthy volunteers
Yes
Inclusion criteria
* Local Health Departments (LHDs, cluster) in the state of Missouri and corresponding public health workforce (individuals within cluster); screenings.
Exclusion criteria
* LHDs that have less than 5 employees working in or supporting diabetes or chronic disease control, which includes program areas of diabetes prevention and management, obesity prevention, physical activity, nutrition, cardiovascular health, and cancer
Design outcomes
Primary
| Measure | Time frame | Description |
|---|---|---|
| Partnerships to Support EBDM | 24 months post baseline | Self-report Likert scale items assess perceived importance of partnering across sectors to share resources and address population health issues. Each item is measured on a 7-point Likert scale from 1 strongly disagree to 7 strongly agree. A summary score was created as an average of the items within the domain. Possible scores 1 to 7. Higher scores mean a better outcome. Items: a) Our collaborative partnerships have missions that align with my agency; b) It is important to my agency to have partners who share resources (money, staff time, space, materials); c) It is important to my agency to have partners in health care to address population health issues; and d) It is important to my agency to have partners in other sectors (outside of health) to address population health issues |
| Evidence-based Decision Making (EBDM) Competencies | 24 months post baseline | Survey participants were asked to rate the perceived importance of each of 10 skills pertinent to evidence-based decision making on an 11-point ordered scale (possible values 0 not at all important to 10 very important for each skill), and to rate the perceived availability in the agency of each of the same skills from 0-10 (0 not at all available to 10 very available). A skill gap was calculated for each skill as perceived importance minus perceived availability (possible values -10 to +10 for each skill). An overall EBDM skill gap was created by taking the average across all 10 skill gaps (possible values -10 to +10). Higher scores indicate a worse outcome. Skill items: community assessment; quantifying the issue; prioritization; action planning; adapting interventions; evaluation designs; quantitative evaluation; qualitative evaluation; economic evaluation; and communicating evidence to decision-makers. A definition for each was provided that started with the word understand. |
| Evidence-based Intervention Score | 24 months post baseline | Self-reported number of evidence-based chronic disease prevention interventions implemented by the local health department from a pre-populated list of 8 evidence-based interventions to prevent diabetes and other chronic diseases (possible score 0 to 8). Higher score indicates a better outcome. |
| Awareness of Culture Supportive of EBDM | 24 months post baseline | Self-report Likert scale items measure personal awareness of opportunities to learn about and apply EBDM among local level chronic disease control public health practitioners. Each item is measured on a 7-point Likert scale from 1 strongly disagree to 7 strongly agree. A summary score was created as an average of the items within the domain. Possible scores 1 to 7. Higher scores mean a better outcome. The items were: a) I am provided the time to identify evidence-based programs and practices; b) My direct supervisor recognizes the value of management practices that facilitate evidence-based decision-making; c) My work group/division offers employees opportunities to attend evidence-based-decision making trainings; and d) Top leadership in my agency (e.g., director, assistant directors) recognizes the value of evidence-based decision-making. |
| Capacity and Expectations for Evidence-based Decision Making (EBDM) | 24 months post baseline | Self-report Likert scale items assess perceived supervisory expectations for EBDM use and perceived work unit/division capacity to carry out EBDM. Each item is measured on a 7-point Likert scale from 1 strongly disagree to 7 strongly agree. A summary score was created as an average of the items (possible scores 1 to 7). Higher scores mean a better outcome. Items: a) I use EBDMin my work; b) My direct supervisor expects me to use evidence-based decision making; c) My performance is partially evaluated on how well I use evidence-based decision making in my work; d) My work group/division currently has the resources (e.g. staff, facilities, partners) to support application of evidence-based decision making; e) The staff in my work group/division has the necessary skills to carry out evidence-based decision making; f) The majority of my work group/division's external partners support use of EBDM; and g) Top leadership in my agency encourages use of EBDM. |
| Resource Availability | 24 months post baseline | Self-report Likert scale items measured perceived work unit's resource availability for evidence-based decision making. Each item is measured on a 7-point Likert scale from 1 strongly disagree to 7 strongly agree. A summary score was created as an average of the items. Possible scores 1 to 7. Higher scores mean a better outcome. The items in the scale were: a) Informational resources (e.g. academic journals, guidelines, and toolkits) are available to my work group/division to promote the use of evidence-based decision making; b) My work group/division engages a diverse external network of partners that share resources to facilitate evidence-based decision making; and c) Stable funding is available for evidence-based decision making. |
| Evaluation Capacity of Work Unit | 24 months post baseline | Self-report Likert scale of work unit's support of community needs assessment, utilization of evaluation for pre and post program implementation as well as for dissemination purposes. Each item is measured on a 7-point Likert scale from 1 strongly disagree to 7 strongly agree. A summary score was created as an average of the items within the domain. Possible scores 1 to 7. Higher scores mean a better outcome. Items: a) My work group/division supports community needs assessments to ensure that evidence-based decision-making approaches continue to meet community needs; b) My work group/division plans for evaluation of interventions prior to implementation; c) My work group/division uses evaluation data to monitor and improve interventions; and d) My work group/division distributes intervention evaluation findings to other organizations that can use our findings. |
| EBDM Climate Cultivation | 24 months post baseline | Self-report Likert scale assessing perceived health department culture supportive of EBDM, information sharing and participatory decision making. Each item is measured on a 7-point Likert scale from 1 strongly disagree to 7 strongly agree. A summary score was created as an average of the items within the domain. Possible scores 1 to 7. Higher scores mean a better outcome. Items: a) My work group/division has access to evidence-based decision making information that is relevant to community needs; b) When decisions are made within my work group/division, program staff members are asked for input; c) Information is widely shared in my work group/division so that everyone who makes decisions has access to all available knowledge; d) My agency is committed to hiring people with relevant training or experience in public health core disciplines (e.g., epidemiology, health education, environmental health); and e) My agency has a culture that supports the processes necessary for EBDM. |
Secondary
| Measure | Time frame | Description |
|---|---|---|
| Inter-agency Connectedness | 24 months post baseline | The average number of links per agency is the measure of connectedness with other agencies that is reported here. The measure is from a separate self-report social network survey. |
Countries
United States
Participant flow
Recruitment details
12 local public health departments were recruited into the study in 3 groups. Once a health department agreed to participate, health department managers provided employee lists for self-report survey invitations.
Pre-assignment details
Each group of 4 health departments crossed over into intervention at different times. In the stepped-wedge design, at baseline, surveys from all 12 health departments were analyzed as control surveys. Once a health department had crossed over into intervention, all remaining surveys of that health department's employees were analyzed as intervention surveys. At the last survey data collection, all surveys were analyzed as intervention surveys.
Participants by arm
| Arm | Count |
|---|---|
| Control The control group will conduct usual public health practice. | 0 |
| Control The control group will conduct usual public health practice. | 433 |
| Intervention Participating local health departments will help develop and choose several dissemination activities they prefer for their local health department to receive. Dissemination activities may include multi-day in-person training workshops, electronic information exchange modalities, remote technical assistance, and information on ways to enhance organizational climates favorable to evidence-based diabetes and chronic disease prevention and control.
Dissemination of public health knowledge: Participating local health departments will help develop and choose dissemination strategies they prefer for their staff working in and supporting diabetes and chronic disease prevention and control to receive. Dissemination strategies may include multi-day in-person training workshops, electronic information exchange modalities, remote technical assistance, and information on ways to enhance organizational climates favorable to evidence-based diabetes prevention and control. | 0 |
| Intervention Participating local health departments will help develop and choose several dissemination activities they prefer for their local health department to receive. Dissemination activities may include multi-day in-person training workshops, electronic information exchange modalities, remote technical assistance, and information on ways to enhance organizational climates favorable to evidence-based diabetes and chronic disease prevention and control.
Dissemination of public health knowledge: Participating local health departments will help develop and choose dissemination strategies they prefer for their staff working in and supporting diabetes and chronic disease prevention and control to receive. Dissemination strategies may include multi-day in-person training workshops, electronic information exchange modalities, remote technical assistance, and information on ways to enhance organizational climates favorable to evidence-based diabetes prevention and control. | 489 |
| Total | 922 |
Withdrawals & dropouts
| Period | Reason | FG000 | FG001 | FG002 |
|---|---|---|---|---|
| Overall Study | Declined survey | 3 | 3 | 2 |
| Overall Study | No longer with agency | 32 | 49 | 30 |
| Overall Study | Not available during data collection | 1 | 0 | 2 |
Baseline characteristics
| Characteristic | Intervention | Total | Control |
|---|---|---|---|
| Age, Customized 18-29 years | 92 completed surveys | 195 completed surveys | 103 completed surveys |
| Age, Customized 30-39 years | 127 completed surveys | 238 completed surveys | 111 completed surveys |
| Age, Customized 40-49 years | 100 completed surveys | 188 completed surveys | 88 completed surveys |
| Age, Customized 50-59 years | 116 completed surveys | 204 completed surveys | 88 completed surveys |
| Age, Customized >=60 years | 52 completed surveys | 94 completed surveys | 42 completed surveys |
| Age, Customized Not reported | 2 completed surveys | 3 completed surveys | 1 completed surveys |
| Race/Ethnicity, Customized | — | 0 completed surveys | — |
| Region of Enrollment United States | 489 completed surveys | 922 completed surveys | 433 completed surveys |
| Sex: Female, Male Female | 411 completed surveys | 767 completed surveys | 356 completed surveys |
| Sex: Female, Male Male | 78 completed surveys | 155 completed surveys | 77 completed surveys |
Adverse events
| Event type | EG000 affected / at risk | EG001 affected / at risk |
|---|---|---|
| deaths Total, all-cause mortality | 0 / 0 | 0 / 0 |
| other Total, other adverse events | 0 / 0 | 0 / 0 |
| serious Total, serious adverse events | 0 / 0 | 0 / 0 |
Outcome results
Primary
Awareness of Culture Supportive of EBDM
Self-report Likert scale items measure personal awareness of opportunities to learn about and apply EBDM among local level chronic disease control public health practitioners. Each item is measured on a 7-point Likert scale from 1 strongly disagree to 7 strongly agree. A summary score was created as an average of the items within the domain. Possible scores 1 to 7. Higher scores mean a better outcome. The items were: a) I am provided the time to identify evidence-based programs and practices; b) My direct supervisor recognizes the value of management practices that facilitate evidence-based decision-making; c) My work group/division offers employees opportunities to attend evidence-based-decision making trainings; and d) Top leadership in my agency (e.g., director, assistant directors) recognizes the value of evidence-based decision-making.
Time frame: 24 months post baseline
Population: The unit of analysis is completed surveys due to the stepped-wedge design and having a different set of survey participants at each of the 4 survey time points. All baseline surveys were analyzed as control; at T4 all surveys were analyzed as intervention. Survey participants were health department employees. Due to staff turnover, some had left the agency between each data collection period and others hired, so additional health department staff were invited at Survey T2, T3, and T4.
| Arm | Measure | Value (MEAN) |
|---|---|---|
| Control | Awareness of Culture Supportive of EBDM | 5.34 score on a scale |
| Intervention | Awareness of Culture Supportive of EBDM | 5.43 score on a scale |
Primary
Capacity and Expectations for Evidence-based Decision Making (EBDM)
Self-report Likert scale items assess perceived supervisory expectations for EBDM use and perceived work unit/division capacity to carry out EBDM. Each item is measured on a 7-point Likert scale from 1 strongly disagree to 7 strongly agree. A summary score was created as an average of the items (possible scores 1 to 7). Higher scores mean a better outcome. Items: a) I use EBDMin my work; b) My direct supervisor expects me to use evidence-based decision making; c) My performance is partially evaluated on how well I use evidence-based decision making in my work; d) My work group/division currently has the resources (e.g. staff, facilities, partners) to support application of evidence-based decision making; e) The staff in my work group/division has the necessary skills to carry out evidence-based decision making; f) The majority of my work group/division's external partners support use of EBDM; and g) Top leadership in my agency encourages use of EBDM.
Time frame: 24 months post baseline
Population: The unit of analysis is completed surveys due to the stepped-wedge design and having a different set of survey participants at each of the 4 survey time points. All baseline surveys were analyzed as control; at T4 all surveys were analyzed as intervention. Survey participants were health department employees. Due to staff turnover, some had left the agency between each data collection period and others hired, so additional health department staff were invited at Survey T2, T3, and T4.
| Arm | Measure | Value (MEAN) |
|---|---|---|
| Control | Capacity and Expectations for Evidence-based Decision Making (EBDM) | 5.22 score on a scale |
| Intervention | Capacity and Expectations for Evidence-based Decision Making (EBDM) | 5.24 score on a scale |
Primary
EBDM Climate Cultivation
Self-report Likert scale assessing perceived health department culture supportive of EBDM, information sharing and participatory decision making. Each item is measured on a 7-point Likert scale from 1 strongly disagree to 7 strongly agree. A summary score was created as an average of the items within the domain. Possible scores 1 to 7. Higher scores mean a better outcome. Items: a) My work group/division has access to evidence-based decision making information that is relevant to community needs; b) When decisions are made within my work group/division, program staff members are asked for input; c) Information is widely shared in my work group/division so that everyone who makes decisions has access to all available knowledge; d) My agency is committed to hiring people with relevant training or experience in public health core disciplines (e.g., epidemiology, health education, environmental health); and e) My agency has a culture that supports the processes necessary for EBDM.
Time frame: 24 months post baseline
Population: The unit of analysis is completed surveys due to the stepped-wedge design and having a different set of survey participants at each of the 4 survey time points. All baseline surveys were analyzed as control; at T4 all surveys were analyzed as intervention. Survey participants were health department employees. Due to staff turnover, some had left the agency between each data collection period and others hired, so additional health department staff were invited at Survey T2, T3, and T4.
| Arm | Measure | Value (MEAN) |
|---|---|---|
| Control | EBDM Climate Cultivation | 5.21 score on a scale |
| Intervention | EBDM Climate Cultivation | 5.26 score on a scale |
Primary
Evaluation Capacity of Work Unit
Self-report Likert scale of work unit's support of community needs assessment, utilization of evaluation for pre and post program implementation as well as for dissemination purposes. Each item is measured on a 7-point Likert scale from 1 strongly disagree to 7 strongly agree. A summary score was created as an average of the items within the domain. Possible scores 1 to 7. Higher scores mean a better outcome. Items: a) My work group/division supports community needs assessments to ensure that evidence-based decision-making approaches continue to meet community needs; b) My work group/division plans for evaluation of interventions prior to implementation; c) My work group/division uses evaluation data to monitor and improve interventions; and d) My work group/division distributes intervention evaluation findings to other organizations that can use our findings.
Time frame: 24 months post baseline
Population: The unit of analysis is completed surveys due to the stepped-wedge design and having a different set of survey participants at each of the 4 survey time points. All baseline surveys were analyzed as control; at T4 all surveys were analyzed as intervention. Survey participants were health department employees. Due to staff turnover, some had left the agency between each data collection period and others hired, so additional health department staff were invited at Survey T2, T3, and T4.
| Arm | Measure | Value (MEAN) |
|---|---|---|
| Control | Evaluation Capacity of Work Unit | 5.23 score on a scale |
| Intervention | Evaluation Capacity of Work Unit | 5.19 score on a scale |
Primary
Evidence-based Decision Making (EBDM) Competencies
Survey participants were asked to rate the perceived importance of each of 10 skills pertinent to evidence-based decision making on an 11-point ordered scale (possible values 0 not at all important to 10 very important for each skill), and to rate the perceived availability in the agency of each of the same skills from 0-10 (0 not at all available to 10 very available). A skill gap was calculated for each skill as perceived importance minus perceived availability (possible values -10 to +10 for each skill). An overall EBDM skill gap was created by taking the average across all 10 skill gaps (possible values -10 to +10). Higher scores indicate a worse outcome. Skill items: community assessment; quantifying the issue; prioritization; action planning; adapting interventions; evaluation designs; quantitative evaluation; qualitative evaluation; economic evaluation; and communicating evidence to decision-makers. A definition for each was provided that started with the word understand.
Time frame: 24 months post baseline
Population: The unit of analysis is completed surveys due to the stepped-wedge design and having a different set of survey participants at each of the 4 survey time points. All baseline surveys were analyzed as control; at T4 all surveys were analyzed as intervention. Survey participants were health department employees. Due to staff turnover, some had left the agency between each data collection period and others hired, so additional health department staff were invited at Survey T2, T3, and T4.
| Arm | Measure | Value (MEAN) |
|---|---|---|
| Control | Evidence-based Decision Making (EBDM) Competencies | 2.05 score on a scale |
| Intervention | Evidence-based Decision Making (EBDM) Competencies | 1.98 score on a scale |
Primary
Evidence-based Intervention Score
Self-reported number of evidence-based chronic disease prevention interventions implemented by the local health department from a pre-populated list of 8 evidence-based interventions to prevent diabetes and other chronic diseases (possible score 0 to 8). Higher score indicates a better outcome.
Time frame: 24 months post baseline
Population: The unit of analysis is completed surveys due to the stepped-wedge design and having a different set of survey participants at each of the 4 survey time points. All baseline surveys were analyzed as control; at T4 all surveys were analyzed as intervention. Survey participants were health department employees. Due to staff turnover, some had left the agency between each data collection period and others hired, so additional health department staff were invited at Survey T2, T3, and T4.
| Arm | Measure | Value (MEAN) |
|---|---|---|
| Control | Evidence-based Intervention Score | 4.84 units on a scale |
| Intervention | Evidence-based Intervention Score | 4.58 units on a scale |
Primary
Partnerships to Support EBDM
Self-report Likert scale items assess perceived importance of partnering across sectors to share resources and address population health issues. Each item is measured on a 7-point Likert scale from 1 strongly disagree to 7 strongly agree. A summary score was created as an average of the items within the domain. Possible scores 1 to 7. Higher scores mean a better outcome. Items: a) Our collaborative partnerships have missions that align with my agency; b) It is important to my agency to have partners who share resources (money, staff time, space, materials); c) It is important to my agency to have partners in health care to address population health issues; and d) It is important to my agency to have partners in other sectors (outside of health) to address population health issues
Time frame: 24 months post baseline
Population: The unit of analysis is completed surveys due to the stepped-wedge design and having a different set of survey participants at each of the 4 survey time points. All baseline surveys were analyzed as control; at T4 all surveys were analyzed as intervention. Survey participants were health department employees. Due to staff turnover, some had left the agency between each data collection period and others hired, so additional health department staff were invited at Survey T2, T3, and T4.
| Arm | Measure | Value (MEAN) |
|---|---|---|
| Control | Partnerships to Support EBDM | 5.94 score on a scale |
| Intervention | Partnerships to Support EBDM | 5.91 score on a scale |
Primary
Resource Availability
Self-report Likert scale items measured perceived work unit's resource availability for evidence-based decision making. Each item is measured on a 7-point Likert scale from 1 strongly disagree to 7 strongly agree. A summary score was created as an average of the items. Possible scores 1 to 7. Higher scores mean a better outcome. The items in the scale were: a) Informational resources (e.g. academic journals, guidelines, and toolkits) are available to my work group/division to promote the use of evidence-based decision making; b) My work group/division engages a diverse external network of partners that share resources to facilitate evidence-based decision making; and c) Stable funding is available for evidence-based decision making.
Time frame: 24 months post baseline
Population: The unit of analysis is completed surveys due to the stepped-wedge design and having a different set of survey participants at each of the 4 survey time points. All baseline surveys were analyzed as control; at T4 all surveys were analyzed as intervention. Survey participants were health department employees. Due to staff turnover, some had left the agency between each data collection period and others hired, so additional health department staff were invited at Survey T2, T3, and T4.
| Arm | Measure | Value (MEAN) |
|---|---|---|
| Control | Resource Availability | 4.46 score on a scale |
| Intervention | Resource Availability | 4.47 score on a scale |
Secondary
Inter-agency Connectedness
The average number of links per agency is the measure of connectedness with other agencies that is reported here. The measure is from a separate self-report social network survey.
Time frame: 24 months post baseline
Population: Agencies with completed social network surveys at baseline when all health departments were in control status and at the end of the study when all health departments were in the intervention.
| Arm | Measure | Value (MEAN) | Dispersion |
|---|---|---|---|
| Control | Inter-agency Connectedness | 6.8 Agencies | Standard Deviation 1.7 |
| Intervention | Inter-agency Connectedness | 8.5 Agencies | Standard Deviation 2.7 |