Influenza
Conditions
Keywords
Vaccination, Health Promotion, Health Behavior, Reminder Systems, Economics, Behavioral, Risk Assessment
Brief summary
The purpose of this study is to assess, prospectively, the effect on flu vaccination rates of salient alerts in the electronic health record that indicate a patient's high risk for flu and its complications. The investigators hypothesize that the salient alerts will lead to increased flu vaccination compared with a standard flu alert.
Detailed description
The CDC (Centers for Disease Control) recommends a flu vaccination to everyone aged 6+ months, with rare exception; almost anyone can benefit from the vaccine, which can reduce illnesses, missed work, hospitalizations, and death. One barrier to vaccination is a lack of cues to action, and, in particular, the lack of direct recommendation from medical personnel; this barrier is arguably the most effectively overcome by a simple nudge of clinicians, compared with barriers such as negative attitudes toward vaccination, low perceived utility of vaccination, and less experience with having received the vaccine. Geisinger partnered with Medial EarlySign (Medial) to develop a machine learning (ML) algorithm to help identify people at risk for serious flu-associated complications based on existing electronic health record data. Eligible at-risk patients will be randomized to an active control group (clinician will be shown a standard flu alert) or one of two experimental groups (clinician will be shown an alert indicating patient's high risk, with or without describing the patient's factors contributing to that risk).
Interventions
BEHAVIORALAlert
Non-interruptive best practice alert in the electronic health record
BEHAVIORALSalient alert features
Larger alert header and body font size, use of different font colors and boldface
BEHAVIORALHigh-risk Text
Alert header indicates patient is at high risk for flu and its complications; alert body indicates the percentage of risk (e.g., in the top 3% of risk)
BEHAVIORALRisk factors
Alert body indicates the top 3 factors contributing to the high risk
Sponsors
Massachusetts Institute of Technology
National Institute on Aging (NIA)
National Bureau of Economic Research, Inc.
Geisinger Clinic
Study design
Allocation
RANDOMIZED
Intervention model
PARALLEL
Primary purpose
PREVENTION
Masking
SINGLE (Subject)
Masking description
Patient-participants will not be explicitly told about the different arms, although clinician-participants will see the different arms and may notice these differences.
Eligibility
Sex/Gender
ALL
Age
18 Years to No maximum
Healthy volunteers
No
Inclusion criteria
Patient Inclusion Criteria: * Age 18+ * Have been determined to be in the top 20% of risk through Medial's ML algorithm * Attend an appointment where the flu alert fires (Geisinger sets when flu alerts start and end--between \ 9/1/2022 and \ 4/30/2023, as well as the trigger conditions for the alert, which includes valid departments and visits and excludes contraindications like Guillain-Barre syndrome) Clinician Inclusion Criteria: * Any Geisinger clinician who sees patient-participants in our study for an appointment where their flu shot alert fires
Design outcomes
Primary
| Measure | Time frame | Description |
|---|---|---|
| Flu Vaccination | At the 1 day visit | Patient received a flu vaccine (yes/no) |
Other
| Measure | Time frame | Description |
|---|---|---|
| High Confidence Flu Diagnosis | Up to 8 months | Patient received a flu diagnosis via a positive polymerase chain reaction \[PCR\]/antigen/molecular test (yes/no) during the 2022-23 flu season (from the patient's appointment date through April 30, 2023) |
| Likely Flu Diagnosis | Up to 8 months | Received a high confidence flu diagnosis (with positive polymerase chain reaction \[PCR\]/antigen/molecular test) and/or likely flu diagnosis (as assessed via International Classification of Disease \[ICD\] codes or Tamiflu administration or positive PCR/antigen/molecular test) (yes/no) during the 2022-23 flu season (from the patient's appointment date through April 30, 2023) Note that likely flu is a superset of the high confidence flu diagnoses. |
| Flu Complications | Up to 8 months | Diagnosed with flu-related complications (yes/no) during the 2022-23 flu season (from the patient's appointment date through April 30, 2023) |
| ER Visits | Up to 11 months | Number of ER visits from the patient's message appointment date through July 31, 2023 |
| COVID-19 Vaccination Rates | Up to 8 months | Received at least one COVID-19 vaccination (yes/no) during the 2022-23 flu season (from the patient's appointment date through April 30, 2023) |
| Flu Vaccination During the 2022-2023 Season | Up to 8 months | Patient receives a flu vaccine (yes/no) during the 2022-23 flu season (from the patient's appointment date through April 30, 2023) |
| Flu Vaccination by Sex | At the 1 day visit | Patient received a flu vaccine (yes/no) |
| Flu Vaccination by Race | At the 1 day visit | Patient received a flu vaccine (yes/no) |
| Flu Vaccination by Ethnicity | At the 1 day visit | Patient received a flu vaccine (yes/no) |
| Hospitalizations | Up to 11 months | Number of hospitalizations from the patient's message appointment date through July 31, 2023 |
Countries
United States
Participant flow
Participants by arm
| Arm | Count |
|---|---|
| Standard Alert Standard flu alert
Alert: Non-interruptive best practice alert in the electronic health record | 24,835 |
| High-risk Alert Flu alert that indicates patient is at high risk for flu and its complications
Alert: Non-interruptive best practice alert in the electronic health record
Salient alert features: Larger alert header and body font size, use of different font colors and boldface
High-risk Text: Alert header indicates patient is at high risk for flu and its complications; alert body indicates the percentage of risk (e.g., in the top 3% of risk) | 24,854 |
| High-risk Alert With Risk Factors Flu alert that indicates patient is at high risk for flu and its complications and presents the factors contributing to this high risk
Alert: Non-interruptive best practice alert in the electronic health record
Salient alert features: Larger alert header and body font size, use of different font colors and boldface
High-risk Text: Alert header indicates patient is at high risk for flu and its complications; alert body indicates the percentage of risk (e.g., in the top 3% of risk)
Risk factors: Alert body indicates the top 3 factors contributing to the high risk | 24,912 |
| Clinicians Geisinger clinician who saw patient-participants in our study for an appointment where their flu shot alert fires | 5,851 |
| Total | 80,452 |
Baseline characteristics
| Characteristic | Standard Alert | High-risk Alert | High-risk Alert With Risk Factors | Clinicians | Total |
|---|---|---|---|---|---|
| Age, Customized >=18 and <65 | 15945 Participants | 15783 Participants | 15936 Participants | 5254 Participants | 52918 Participants |
| Age, Customized >65 | 8890 Participants | 9071 Participants | 8976 Participants | 300 Participants | 27237 Participants |
| Age, Customized Unknown or not reported | 0 Participants | 0 Participants | 0 Participants | 297 Participants | 297 Participants |
| Ethnicity (NIH/OMB) Hispanic or Latino | 1108 Participants | 1201 Participants | 1160 Participants | 165 Participants | 3634 Participants |
| Ethnicity (NIH/OMB) Not Hispanic or Latino | 23571 Participants | 23503 Participants | 23605 Participants | 5276 Participants | 75955 Participants |
| Ethnicity (NIH/OMB) Unknown or Not Reported | 156 Participants | 150 Participants | 147 Participants | 410 Participants | 863 Participants |
| Race/Ethnicity, Customized Race American Indian or Alaska Native | 55 Participants | 58 Participants | 60 Participants | 4 Participants | 177 Participants |
| Race/Ethnicity, Customized Race Asian | 68 Participants | 72 Participants | 83 Participants | 397 Participants | 620 Participants |
| Race/Ethnicity, Customized Race Black or African American | 871 Participants | 837 Participants | 919 Participants | 129 Participants | 2756 Participants |
| Race/Ethnicity, Customized Race Native Hawaiian or Other Pacific Islander | 71 Participants | 66 Participants | 93 Participants | 5 Participants | 235 Participants |
| Race/Ethnicity, Customized Race Other | 11 Participants | 12 Participants | 15 Participants | 0 Participants | 38 Participants |
| Race/Ethnicity, Customized Race Two or More | 4 Participants | 5 Participants | 5 Participants | 55 Participants | 69 Participants |
| Race/Ethnicity, Customized Race Unknown or Not Reported | 142 Participants | 153 Participants | 148 Participants | 575 Participants | 1018 Participants |
| Race/Ethnicity, Customized Race White | 23613 Participants | 23651 Participants | 23589 Participants | 4686 Participants | 75539 Participants |
| Region of Enrollment United States | 24835 participants | 24854 participants | 24912 participants | 5851 participants | 80452 participants |
| Sex/Gender, Customized Female | 16334 Participants | 16445 Participants | 16368 Participants | 4224 Participants | 53371 Participants |
| Sex/Gender, Customized Male | 8501 Participants | 8409 Participants | 8544 Participants | 1322 Participants | 26776 Participants |
| Sex/Gender, Customized Unknown or not reported | 0 Participants | 0 Participants | 0 Participants | 305 Participants | 305 Participants |
Adverse events
| Event type | EG000 affected / at risk | EG001 affected / at risk | EG002 affected / at risk |
|---|---|---|---|
| deaths Total, all-cause mortality | 0 / 0 | 0 / 0 | 0 / 0 |
| other Total, other adverse events | 0 / 0 | 0 / 0 | 0 / 0 |
| serious Total, serious adverse events | 0 / 0 | 0 / 0 | 0 / 0 |
Outcome results
Primary
Flu Vaccination
Patient received a flu vaccine (yes/no)
Time frame: At the 1 day visit
Population: As preregistered, our primary outcome analysis is limited to the first in-person appointment for each patient enrolled in the study. Therefore, this analysis excludes the patients who only attended telehealth appointments during the study.
| Arm | Measure | Value (COUNT_OF_PARTICIPANTS) |
|---|---|---|
| Standard Alert | Flu Vaccination | 4459 Participants |
| High-risk Alert | Flu Vaccination | 4456 Participants |
| High-risk Alert With Risk Factors | Flu Vaccination | 4575 Participants |
Comparison: Null hypothesis: there is no difference in flu shots for patients whose clinicians were shown alerts with information about their risk status (patients randomized to the High-risk Alert or High-risk Alert with Risk Factors arms) compared with those whose clinicians were shown the standard alert. Alternative hypothesis: patients in the High-risk Alert and High-risk Alert with Risk Factors arms will exhibit improved flu vaccination rates compared with those in the standard Alert arm.p-value: 0.517Regression, Linear
Comparison: Null hypothesis: there is no difference in flu shots for patients whose clinicians were shown alerts the factors that contributed to a patient's high risk (High-risk Alert with Risk Factors arm) compared with those whose clinicians were shown the alert with risk level only (High-risk Alert arm). Alternative hypothesis: patients in the High-risk Alert with Risk Factors arm will exhibit improved flu vaccination rates compared with those in the High-risk Alert arm.p-value: 0.226Regression, Linear
Other Pre-specified
COVID-19 Vaccination Rates
Received at least one COVID-19 vaccination (yes/no) during the 2022-23 flu season (from the patient's appointment date through April 30, 2023)
Time frame: Up to 8 months
Other Pre-specified
ER Visits
Number of ER visits from the patient's message appointment date through July 31, 2023
Time frame: Up to 11 months
Other Pre-specified
Flu Complications
Diagnosed with flu-related complications (yes/no) during the 2022-23 flu season (from the patient's appointment date through April 30, 2023)
Time frame: Up to 8 months
Other Pre-specified
Flu Vaccination by Ethnicity
Patient received a flu vaccine (yes/no)
Time frame: At the 1 day visit
Population: As preregistered, our primary outcome analysis is limited to the first in-person appointment for each patient enrolled in the study. Therefore, this analysis excludes the patients who only attended telehealth appointments during the study.
| Arm | Measure | Group | Value (COUNT_OF_PARTICIPANTS) |
|---|---|---|---|
| Standard Alert | Flu Vaccination by Ethnicity | Unknown or Not Reported | 17 Participants |
| Standard Alert | Flu Vaccination by Ethnicity | Not Hispanic or Latino | 4278 Participants |
| Standard Alert | Flu Vaccination by Ethnicity | Hispanic or Latino | 164 Participants |
| High-risk Alert | Flu Vaccination by Ethnicity | Unknown or Not Reported | 28 Participants |
| High-risk Alert | Flu Vaccination by Ethnicity | Hispanic or Latino | 184 Participants |
| High-risk Alert | Flu Vaccination by Ethnicity | Not Hispanic or Latino | 4244 Participants |
| High-risk Alert With Risk Factors | Flu Vaccination by Ethnicity | Unknown or Not Reported | 26 Participants |
| High-risk Alert With Risk Factors | Flu Vaccination by Ethnicity | Not Hispanic or Latino | 4377 Participants |
| High-risk Alert With Risk Factors | Flu Vaccination by Ethnicity | Hispanic or Latino | 172 Participants |
Other Pre-specified
Flu Vaccination by Race
Patient received a flu vaccine (yes/no)
Time frame: At the 1 day visit
Population: As preregistered, our primary outcome analysis is limited to the first in-person appointment for each patient enrolled in the study. Therefore, this analysis excludes the patients who only attended telehealth appointments during the study.
| Arm | Measure | Group | Value (COUNT_OF_PARTICIPANTS) |
|---|---|---|---|
| Standard Alert | Flu Vaccination by Race | Native Hawaiian or Other Pacific Islander | 10 Participants |
| Standard Alert | Flu Vaccination by Race | White | 4302 Participants |
| Standard Alert | Flu Vaccination by Race | Unknown or Not Reported | 24 Participants |
| Standard Alert | Flu Vaccination by Race | Other | 3 Participants |
| Standard Alert | Flu Vaccination by Race | Black or African American | 94 Participants |
| Standard Alert | Flu Vaccination by Race | Two or More | 0 Participants |
| Standard Alert | Flu Vaccination by Race | American Indian or Alaska Native | 11 Participants |
| Standard Alert | Flu Vaccination by Race | Asian | 15 Participants |
| High-risk Alert | Flu Vaccination by Race | Black or African American | 90 Participants |
| High-risk Alert | Flu Vaccination by Race | Native Hawaiian or Other Pacific Islander | 12 Participants |
| High-risk Alert | Flu Vaccination by Race | White | 4306 Participants |
| High-risk Alert | Flu Vaccination by Race | Other | 3 Participants |
| High-risk Alert | Flu Vaccination by Race | Two or More | 1 Participants |
| High-risk Alert | Flu Vaccination by Race | Asian | 20 Participants |
| High-risk Alert | Flu Vaccination by Race | Unknown or Not Reported | 19 Participants |
| High-risk Alert | Flu Vaccination by Race | American Indian or Alaska Native | 5 Participants |
| High-risk Alert With Risk Factors | Flu Vaccination by Race | Unknown or Not Reported | 26 Participants |
| High-risk Alert With Risk Factors | Flu Vaccination by Race | Two or More | 1 Participants |
| High-risk Alert With Risk Factors | Flu Vaccination by Race | Native Hawaiian or Other Pacific Islander | 17 Participants |
| High-risk Alert With Risk Factors | Flu Vaccination by Race | Black or African American | 125 Participants |
| High-risk Alert With Risk Factors | Flu Vaccination by Race | White | 4372 Participants |
| High-risk Alert With Risk Factors | Flu Vaccination by Race | American Indian or Alaska Native | 9 Participants |
| High-risk Alert With Risk Factors | Flu Vaccination by Race | Asian | 21 Participants |
| High-risk Alert With Risk Factors | Flu Vaccination by Race | Other | 4 Participants |
Other Pre-specified
Flu Vaccination by Sex
Patient received a flu vaccine (yes/no)
Time frame: At the 1 day visit
Population: As preregistered, our primary outcome analysis is limited to the first in-person appointment for each patient enrolled in the study. Therefore, this analysis excludes the patients who only attended telehealth appointments during the study.
| Arm | Measure | Group | Value (COUNT_OF_PARTICIPANTS) |
|---|---|---|---|
| Standard Alert | Flu Vaccination by Sex | Female | 2817 Participants |
| Standard Alert | Flu Vaccination by Sex | Male | 1642 Participants |
| High-risk Alert | Flu Vaccination by Sex | Female | 2766 Participants |
| High-risk Alert | Flu Vaccination by Sex | Male | 1690 Participants |
| High-risk Alert With Risk Factors | Flu Vaccination by Sex | Female | 2894 Participants |
| High-risk Alert With Risk Factors | Flu Vaccination by Sex | Male | 1681 Participants |
Other Pre-specified
Flu Vaccination During the 2022-2023 Season
Patient receives a flu vaccine (yes/no) during the 2022-23 flu season (from the patient's appointment date through April 30, 2023)
Time frame: Up to 8 months
Other Pre-specified
High Confidence Flu Diagnosis
Patient received a flu diagnosis via a positive polymerase chain reaction \[PCR\]/antigen/molecular test (yes/no) during the 2022-23 flu season (from the patient's appointment date through April 30, 2023)
Time frame: Up to 8 months
Other Pre-specified
Hospitalizations
Number of hospitalizations from the patient's message appointment date through July 31, 2023
Time frame: Up to 11 months
Other Pre-specified
Likely Flu Diagnosis
Received a high confidence flu diagnosis (with positive polymerase chain reaction \[PCR\]/antigen/molecular test) and/or likely flu diagnosis (as assessed via International Classification of Disease \[ICD\] codes or Tamiflu administration or positive PCR/antigen/molecular test) (yes/no) during the 2022-23 flu season (from the patient's appointment date through April 30, 2023) Note that likely flu is a superset of the high confidence flu diagnoses.
Time frame: Up to 8 months