Influenza
Conditions
Keywords
Vaccination, Health promotion, Health behavior, Economics, Behavioral
Brief summary
The purpose of this study is to test whether messages that make it easy to schedule a flu shot appointment will increase flu shot rates in patients without an upcoming appointment. The study will also test which message versions and message timing are most effective for increasing flu vaccination.
Detailed description
The CDC recommends a flu vaccination to everyone aged 6+ months, with rare exception; almost anyone can benefit from the vaccine, which can reduce illnesses, missed work, hospitalizations, and death. Past work from the study team focused on encouraging flu shots for patients with upcoming appointments. However, many patients in the health system do not have any appointments during flu season. Eligible patients without an upcoming appointment will be randomized to a passive control group (no message), an active control group (a basic message stating that the patient can get a flu shot at Geisinger) or one of several other messages informed by behavioral science (ease, waiting for you, protect yourself - rare outcomes, or protect yourself - frequent outcomes). Patients will be randomly assigned to one of several message send dates. Messages sent via patient portal, short message service (SMS) text, email, and/or another modality, will include a link redirecting patients to a page where they can self-schedule a flu shot.
Interventions
BEHAVIORALReminder
Patient portal, SMS, email, and/or another modality
Sponsors
Massachusetts Institute of Technology
National Bureau of Economic Research, Inc.
Geisinger Clinic
Study design
Allocation
RANDOMIZED
Intervention model
PARALLEL
Primary purpose
PREVENTION
Masking
NONE
Masking description
Although patients will not be explicitly informed of which arm they were randomized to, they will be aware of the messages they receive.
Eligibility
Sex/Gender
ALL
Age
18 Years to No maximum
Healthy volunteers
No
Inclusion criteria
* Included in one or both of the following two base patient lists: 1. Geisinger patient portal flu outreach list for Fall 2022 2. A list of patients obtained from Geisinger's Phenomics and Clinical Data Core (PACDC). This list will include active Geisinger patients (all patients on this list attended at least one primary care appointment at Geisinger between 10/1/2008 and 4/13/2022, and either had a Geisinger primary care provider assigned as of April 2022, or were in the Electronic Health Record \[EHR\] since at least September 2021 and had at least one encounter in 2020-2022). * Aged 18 or older as of October 2022 * Has had a Geisinger encounter in the last 2 years as of October 2022 * Either of the following, as of October 2022: 1. Has a Geisinger PCP assigned in the Community Medicine, Pediatrics, or Internal Medicine service line 2. In the last 2 years, has completed an appointment in a Geisinger specialty on a list of specialties approved by system leadership for flu shot communications * Has not received a flu-shot during the 2022-23 flu season as of \ 1 week prior to the message date (timeline may be slightly different, depending on data pull logistics), according to the Electronic Health Record (EHR) * Does not have a scheduled in-person primary care or in-person flu-shot-eligible specialty appointment in the 12 weeks following their assigned message send date, as of \ 1 week prior to the message date (timelines may be slightly different, depending on data pull logistics or clinical guidance) * As of this writing, the team plans to define a flu-shot-eligible specialty appointments as appointments in departments that have historically documented or administered flu vaccine. However, this approach may change slightly based on changing clinical guidance.
Exclusion criteria
* The CMSL/Marketing list will include patients in Geisinger's 65Forward or Community Care populations. If necessary due to logistical constraints, we may exclude these 65Forward and/or Community Care patients from our study. * Cannot be contacted via the communication modality being used in the study (e.g., patient portal, SMS), e.g., due to insufficient/missing contact information in the EHR or because they opted out * Has an allergy to flu vaccines according to any EHR allergy table known to the study team * Has a health maintenance modifier indicating they are permanently discontinued from receiving a seasonal flu shot * Is on a list of dismissed patients
Design outcomes
Primary
| Measure | Time frame | Description |
|---|---|---|
| Number of Patients Who Self-scheduled a Flu Shot | Within 4 weeks of message send date | Patient self-schedules a flu vaccine appointment through the patient portal. This outcome measures whether the patient schedules an appointment within 4 weeks of the send date, but the appointment can occur more than 4 weeks past the send date. |
Secondary
| Measure | Time frame | Description |
|---|---|---|
| Flu Vaccination | Within 4 weeks of message send date | Number of patients who received a flu vaccine |
Other
| Measure | Time frame | Description |
|---|---|---|
| Flu Complications | Up to 10 months | Diagnosed with flu-related complications (yes/no) from the patient's message send date through July 31, 2023. |
| Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Within 4 weeks of the message send date | Patient self-schedules a flu vaccine through the patient portal This outcome measures whether the patient schedules an appointment within 4 weeks of the message send date, but the appointment can occur more than 4 weeks past the send date. |
| ER Visits | Up to 10 months | Number of ER visits from the patient's message send date through July 31, 2023. |
| High Confidence Flu Diagnosis | Up to 7 months | Patient received a flu diagnosis via a positive PCR/antigen/molecular test (yes/no) during the 2022-23 flu season (from the patient's message send date through April 30, 2023). |
| COVID-19 Vaccination Rates | Up to 7 months | Received at least one COVID-19 vaccination (yes/no) during the 2022-23 flu season (from the patient's message send date through April 30, 2023). |
| Number of Patients Who Self-Scheduled a Flu Shot by Sex | Within 4 weeks of the message send date | Patient self-schedules a flu vaccine through the patient portal This outcome measures whether the patient schedules an appointment within 4 weeks of the message send date, but the appointment can occur more than 4 weeks past the send date. |
| Number of Patients Who Self-Scheduled a Flu Shot by Race | Within 4 weeks of the message send date | Patient self-schedules a flu vaccine through the patient portal This outcome measures whether the patient schedules an appointment within 4 weeks of the message send date, but the appointment can occur more than 4 weeks past the send date. |
| Hospitalizations | Up to 10 months | Number of hospitalizations from the patient's message send date through July 31, 2023. |
| Likely Flu Diagnosis | Up to 7 months | Received a high confidence flu diagnosis (with positive polymerase chain reaction \[PCR\]/antigen/molecular test) and/or likely flu diagnosis (as assessed via International Classification of Disease \[ICD\] codes or Tamiflu administration or positive PCR/antigen/molecular test) (yes/no) during the 2022-23 flu season (from the patient's message send date through April 30, 2023). Note that likely flu is a superset of the high confidence flu diagnoses. |
Countries
United States
Participant flow
Participants by arm
| Arm | Count |
|---|---|
| Passive Control - November No message | 12,531 |
| Active Control Message - November A message that simply states that patients can get a flu shot at Geisinger
Reminder: Patient portal, SMS, email, and/or another modality | 12,531 |
| Ease Message - November A message emphasizing the ease of scheduling a flu shot at Geisinger
Reminder: Patient portal, SMS, email, and/or another modality | 12,531 |
| Waiting for You Message - November A message that states the patient's flu shot is waiting for them at Geisinger
Reminder: Patient portal, SMS, email, and/or another modality | 12,531 |
| Protect Yourself - Rare Message - November A message that emphasizes the rare, dangerous outcomes of getting the flu (e.g., hospitalization, pneumonia), and states that a flu shot can offer protection from those outcomes
Reminder: Patient portal, SMS, email, and/or another modality | 12,530 |
| Protect Yourself - Frequent Message - November A message that emphasizes the outcomes that frequently occur in people with the flu (e.g., fever, chills, missing important events), and states that a flu shot can offer protection from those outcomes
Reminder: Patient portal, SMS, email, and/or another modality | 12,531 |
| Passive Control - December No message | 10,705 |
| Active Control Message - December A message that simply states that patients can get a flu shot at Geisinger
Reminder: Patient portal, SMS, email, and/or another modality | 10,743 |
| Ease Message - December A message emphasizing the ease of scheduling a flu shot at Geisinger
Reminder: Patient portal, SMS, email, and/or another modality | 10,657 |
| Waiting for You Message - December A message that states the patient's flu shot is waiting for them at Geisinger
Reminder: Patient portal, SMS, email, and/or another modality | 10,783 |
| Protect Yourself - Rare Message - December A message that emphasizes the rare, dangerous outcomes of getting the flu (e.g., hospitalization, pneumonia), and states that a flu shot can offer protection from those outcomes
Reminder: Patient portal, SMS, email, and/or another modality | 10,702 |
| Protect Yourself - Frequent Message - December A message that emphasizes the outcomes that frequently occur in people with the flu (e.g., fever, chills, missing important events), and states that a flu shot can offer protection from those outcomes
Reminder: Patient portal, SMS, email, and/or another modality | 10,728 |
| Total | 139,503 |
Baseline characteristics
| Characteristic | Passive Control - November | Active Control Message - November | Ease Message - November | Waiting for You Message - November | Protect Yourself - Rare Message - November | Protect Yourself - Frequent Message - November | Passive Control - December | Active Control Message - December | Ease Message - December | Waiting for You Message - December | Protect Yourself - Rare Message - December | Protect Yourself - Frequent Message - December | Total |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Age, Customized 18-64 years | 11105 Participants | 11041 Participants | 11096 Participants | 11073 Participants | 11018 Participants | 11040 Participants | 9573 Participants | 9585 Participants | 9564 Participants | 9621 Participants | 9532 Participants | 9564 Participants | 123812 Participants |
| Age, Customized 65+ years | 1426 Participants | 1490 Participants | 1435 Participants | 1458 Participants | 1512 Participants | 1491 Participants | 1132 Participants | 1158 Participants | 1093 Participants | 1162 Participants | 1170 Participants | 1164 Participants | 15691 Participants |
| Ethnicity (NIH/OMB) Hispanic or Latino | 551 Participants | 581 Participants | 632 Participants | 613 Participants | 590 Participants | 612 Participants | 501 Participants | 513 Participants | 537 Participants | 537 Participants | 528 Participants | 494 Participants | 6689 Participants |
| Ethnicity (NIH/OMB) Not Hispanic or Latino | 11851 Participants | 11810 Participants | 11759 Participants | 11783 Participants | 11794 Participants | 11784 Participants | 10078 Participants | 10119 Participants | 10000 Participants | 10127 Participants | 10060 Participants | 10126 Participants | 131291 Participants |
| Ethnicity (NIH/OMB) Unknown or Not Reported | 129 Participants | 140 Participants | 140 Participants | 135 Participants | 146 Participants | 135 Participants | 126 Participants | 111 Participants | 120 Participants | 119 Participants | 114 Participants | 108 Participants | 1523 Participants |
| Race (NIH/OMB) American Indian or Alaska Native | 21 Participants | 20 Participants | 29 Participants | 41 Participants | 27 Participants | 35 Participants | 25 Participants | 30 Participants | 19 Participants | 33 Participants | 20 Participants | 22 Participants | 322 Participants |
| Race (NIH/OMB) Asian | 159 Participants | 195 Participants | 183 Participants | 181 Participants | 154 Participants | 173 Participants | 128 Participants | 145 Participants | 131 Participants | 139 Participants | 157 Participants | 146 Participants | 1891 Participants |
| Race (NIH/OMB) Black or African American | 508 Participants | 505 Participants | 542 Participants | 507 Participants | 484 Participants | 534 Participants | 460 Participants | 444 Participants | 466 Participants | 450 Participants | 451 Participants | 422 Participants | 5773 Participants |
| Race (NIH/OMB) More than one race | 4 Participants | 5 Participants | 3 Participants | 4 Participants | 1 Participants | 1 Participants | 3 Participants | 0 Participants | 2 Participants | 0 Participants | 0 Participants | 1 Participants | 24 Participants |
| Race (NIH/OMB) Native Hawaiian or Other Pacific Islander | 50 Participants | 56 Participants | 57 Participants | 64 Participants | 50 Participants | 48 Participants | 52 Participants | 62 Participants | 51 Participants | 46 Participants | 37 Participants | 54 Participants | 627 Participants |
| Race (NIH/OMB) Unknown or Not Reported | 153 Participants | 146 Participants | 142 Participants | 168 Participants | 168 Participants | 148 Participants | 123 Participants | 125 Participants | 131 Participants | 135 Participants | 157 Participants | 121 Participants | 1717 Participants |
| Race (NIH/OMB) White | 11636 Participants | 11604 Participants | 11575 Participants | 11566 Participants | 11646 Participants | 11592 Participants | 9914 Participants | 9937 Participants | 9857 Participants | 9980 Participants | 9880 Participants | 9962 Participants | 129149 Participants |
| Region of Enrollment United States | 12531 participants | 12531 participants | 12531 participants | 12531 participants | 12530 participants | 12531 participants | 10705 participants | 10743 participants | 10657 participants | 10783 participants | 10702 participants | 10728 participants | 139503 participants |
| Sex: Female, Male Female | 7019 Participants | 7025 Participants | 7048 Participants | 7126 Participants | 7047 Participants | 7153 Participants | 6002 Participants | 5964 Participants | 5945 Participants | 6007 Participants | 5960 Participants | 6072 Participants | 78368 Participants |
| Sex: Female, Male Male | 5512 Participants | 5506 Participants | 5483 Participants | 5405 Participants | 5483 Participants | 5378 Participants | 4703 Participants | 4779 Participants | 4712 Participants | 4776 Participants | 4742 Participants | 4656 Participants | 61135 Participants |
Adverse events
| Event type | EG000 affected / at risk | EG001 affected / at risk | EG002 affected / at risk | EG003 affected / at risk | EG004 affected / at risk | EG005 affected / at risk | EG006 affected / at risk | EG007 affected / at risk | EG008 affected / at risk | EG009 affected / at risk | EG010 affected / at risk | EG011 affected / at risk |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| deaths Total, all-cause mortality | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 |
| other Total, other adverse events | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 |
| serious Total, serious adverse events | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 |
Outcome results
Primary
Number of Patients Who Self-scheduled a Flu Shot
Patient self-schedules a flu vaccine appointment through the patient portal. This outcome measures whether the patient schedules an appointment within 4 weeks of the send date, but the appointment can occur more than 4 weeks past the send date.
Time frame: Within 4 weeks of message send date
| Arm | Measure | Value (COUNT_OF_PARTICIPANTS) |
|---|---|---|
| Passive Control - November | Number of Patients Who Self-scheduled a Flu Shot | 43 Participants |
| Active Control Message - November | Number of Patients Who Self-scheduled a Flu Shot | 63 Participants |
| Ease Message - November | Number of Patients Who Self-scheduled a Flu Shot | 62 Participants |
| Waiting for You Message - November | Number of Patients Who Self-scheduled a Flu Shot | 74 Participants |
| Protect Yourself - Rare Message | Number of Patients Who Self-scheduled a Flu Shot | 70 Participants |
| Protect Yourself - Frequent Message - November | Number of Patients Who Self-scheduled a Flu Shot | 53 Participants |
| Passive Control - December | Number of Patients Who Self-scheduled a Flu Shot | 8 Participants |
| Active Control Message - December | Number of Patients Who Self-scheduled a Flu Shot | 28 Participants |
| Ease Message - December | Number of Patients Who Self-scheduled a Flu Shot | 36 Participants |
| Waiting for You Message - December | Number of Patients Who Self-scheduled a Flu Shot | 44 Participants |
| Protect Yourself - Rare Message - December | Number of Patients Who Self-scheduled a Flu Shot | 20 Participants |
| Protect Yourself - Frequent Message - December | Number of Patients Who Self-scheduled a Flu Shot | 26 Participants |
Comparison: Null hypothesis: Flu shot self-scheduling does not differ between those sent an Active Control message and those not sent a message (i.e., those in the Passive Control group); Alternative hypothesis: Sending an Active Control message increases self-scheduling compared with sending no message. Analysis combines across November and December send dates.p-value: 0.003Regression, Linear
Comparison: Null hypothesis: Flu shot self-scheduling does not differ between those sent an Ease message and those not sent a message (i.e., those in the Passive Control group); Alternative hypothesis: Sending an Ease message increases self-scheduling compared with sending no message. Analysis combines across November and December send dates.p-value: <0.001Regression, Linear
Comparison: Null hypothesis: Flu shot self-scheduling does not differ between those sent an Waiting for You message message and those not sent a message (i.e., those in the Passive Control group); Alternative hypothesis: Sending a Waiting for You message increases self-scheduling compared with sending no message. Analysis combines across November and December send dates.p-value: <0.001Regression, Linear
Comparison: Null hypothesis: Flu shot self-scheduling does not differ between those sent a Protect Yourself - Rare message and those not sent a message (i.e., those in the Passive Control group); Alternative hypothesis: Sending a Protect Yourself - Rare Message increases self-scheduling compared with sending no message. Analysis combines across November and December send dates.p-value: 0.003Regression, Linear
Comparison: Null hypothesis: Flu shot self-scheduling does not differ between those sent a Protect Yourself - Frequent message and those not sent a message (i.e., those in the Passive Control group); Alternative hypothesis: Sending a Protect Yourself - Frequent message increases self-scheduling compared with sending no message. Analysis combines across November and December send dates.p-value: 0.035Regression, Linear
Comparison: Null hypothesis: Active Control messages and Ease messages are equally effective at promoting flu shot self-scheduling; Alternative hypothesis: Ease messages are more effective at promoting flu shot self-scheduling than Active Control messages. Analysis combines across November and December send dates.p-value: 0.593Regression, Linear
Comparison: Null hypothesis: Active Control messages and Waiting for You messages are equally effective at promoting flu shot self-scheduling; Alternative hypothesis: Waiting for You messages are more effective at promoting flu shot self-scheduling than Active Control messages. Analysis combines across November and December send dates.p-value: 0.052Regression, Linear
Comparison: Null hypothesis: Active Control messages and Protect Yourself - Rare messages are equally effective at promoting flu shot self-scheduling; Alternative hypothesis: Protect Yourself - Rare messages are more effective at promoting flu shot self-scheduling than Active Control messages. Analysis combines across November and December send dates.p-value: 0.952Regression, Linear
Comparison: Null hypothesis: Active Control messages and Protect Yourself - Frequent messages are equally effective at promoting flu shot self-scheduling; Alternative hypothesis: Protect Yourself - Frequent messages are more effective at promoting flu shot self-scheduling than Active Control messages. Analysis combines across November and December send dates.p-value: 0.386Regression, Linear
Comparison: Null hypothesis: Ease messages and Waiting for You messages are equally effective at promoting flu shot self-scheduling; Alternative hypothesis: There is a difference in effectiveness between Ease and Waiting for You messages. Analysis combines across November and December send dates.p-value: 0.498Regression, Linear
Comparison: Null hypothesis: Ease messages and Protect Yourself - Rare messages are equally effective at promoting flu shot self-scheduling; Alternative hypothesis: There is a difference in effectiveness between Ease and Protect Yourself - Rare messages. Analysis combines across November and December send dates.p-value: 0.935Regression, Linear
Comparison: Null hypothesis: Ease messages and Protect Yourself - Frequent messages are equally effective at promoting flu shot self-scheduling; Alternative hypothesis: There is a difference in effectiveness between Ease and Protect Yourself - Frequent message messages. Analysis combines across November and December send dates.p-value: 0.504Regression, Linear
Comparison: Null hypothesis: Protect Yourself - Rare and Waiting for You messages are equally effective at promoting flu shot self-scheduling; Alternative hypothesis: There is a difference in effectiveness between Protect Yourself - Rare and Waiting for You messages. Analysis combines across November and December send dates.p-value: 0.19Regression, Linear
Comparison: Null hypothesis: Protect Yourself - Frequent messages and Waiting for You messages are equally effective at promoting flu shot self-scheduling; Alternative hypothesis: There is a difference in effectiveness between Protect Yourself - Frequent and Waiting for You messages. Analysis combines across November and December send dates.p-value: 0.027Regression, Linear
Comparison: Null hypothesis: Protect Yourself - Rare and Protect Yourself - Frequent messages are equally effective at promoting flu shot self-scheduling; Alternative hypothesis: There is a difference in effectiveness between Protect Yourself - Rare and Protect Yourself - Frequent messages. Analysis combines across November and December send dates.p-value: 0.854Regression, Linear
Comparison: Null Hypothesis: Messages sent in November and December are equally effective at promoting flu-shot self-scheduling. Alternative hypothesis: Messages sent in November and December are differentially effective.~To test the alternative hypothesis, all message arms (Active control, Ease, Waiting for you, Protect yourself - rare, Protect yourself - frequent) were combined and compared with Passive control.p-value: 0.637Regression, Linear
Secondary
Flu Vaccination
Number of patients who received a flu vaccine
Time frame: Within 4 weeks of message send date
| Arm | Measure | Value (COUNT_OF_PARTICIPANTS) |
|---|---|---|
| Passive Control - November | Flu Vaccination | 488 Participants |
| Active Control Message - November | Flu Vaccination | 450 Participants |
| Ease Message - November | Flu Vaccination | 468 Participants |
| Waiting for You Message - November | Flu Vaccination | 460 Participants |
| Protect Yourself - Rare Message | Flu Vaccination | 450 Participants |
| Protect Yourself - Frequent Message - November | Flu Vaccination | 478 Participants |
| Passive Control - December | Flu Vaccination | 146 Participants |
| Active Control Message - December | Flu Vaccination | 158 Participants |
| Ease Message - December | Flu Vaccination | 153 Participants |
| Waiting for You Message - December | Flu Vaccination | 177 Participants |
| Protect Yourself - Rare Message - December | Flu Vaccination | 142 Participants |
| Protect Yourself - Frequent Message - December | Flu Vaccination | 124 Participants |
Other Pre-specified
COVID-19 Vaccination Rates
Received at least one COVID-19 vaccination (yes/no) during the 2022-23 flu season (from the patient's message send date through April 30, 2023).
Time frame: Up to 7 months
Other Pre-specified
ER Visits
Number of ER visits from the patient's message send date through July 31, 2023.
Time frame: Up to 10 months
Other Pre-specified
Flu Complications
Diagnosed with flu-related complications (yes/no) from the patient's message send date through July 31, 2023.
Time frame: Up to 10 months
Other Pre-specified
High Confidence Flu Diagnosis
Patient received a flu diagnosis via a positive PCR/antigen/molecular test (yes/no) during the 2022-23 flu season (from the patient's message send date through April 30, 2023).
Time frame: Up to 7 months
Other Pre-specified
Hospitalizations
Number of hospitalizations from the patient's message send date through July 31, 2023.
Time frame: Up to 10 months
Other Pre-specified
Likely Flu Diagnosis
Received a high confidence flu diagnosis (with positive polymerase chain reaction \[PCR\]/antigen/molecular test) and/or likely flu diagnosis (as assessed via International Classification of Disease \[ICD\] codes or Tamiflu administration or positive PCR/antigen/molecular test) (yes/no) during the 2022-23 flu season (from the patient's message send date through April 30, 2023). Note that likely flu is a superset of the high confidence flu diagnoses.
Time frame: Up to 7 months
Other Pre-specified
Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity
Patient self-schedules a flu vaccine through the patient portal This outcome measures whether the patient schedules an appointment within 4 weeks of the message send date, but the appointment can occur more than 4 weeks past the send date.
Time frame: Within 4 weeks of the message send date
Population: This outcome divides the population by ethnicity (as listed in the electronic health record)
| Arm | Measure | Group | Value (COUNT_OF_PARTICIPANTS) |
|---|---|---|---|
| Passive Control - November | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Unable to Obtain | 0 Participants |
| Passive Control - November | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Hispanic or Latino | 3 Participants |
| Passive Control - November | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Not Hispanic or Latino | 40 Participants |
| Active Control Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Not Hispanic or Latino | 61 Participants |
| Active Control Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Unable to Obtain | 0 Participants |
| Active Control Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Hispanic or Latino | 2 Participants |
| Ease Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Hispanic or Latino | 3 Participants |
| Ease Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Not Hispanic or Latino | 59 Participants |
| Ease Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Unable to Obtain | 0 Participants |
| Waiting for You Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Not Hispanic or Latino | 69 Participants |
| Waiting for You Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Unable to Obtain | 0 Participants |
| Waiting for You Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Hispanic or Latino | 5 Participants |
| Protect Yourself - Rare Message | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Hispanic or Latino | 0 Participants |
| Protect Yourself - Rare Message | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Unable to Obtain | 1 Participants |
| Protect Yourself - Rare Message | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Not Hispanic or Latino | 69 Participants |
| Protect Yourself - Frequent Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Unable to Obtain | 1 Participants |
| Protect Yourself - Frequent Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Not Hispanic or Latino | 51 Participants |
| Protect Yourself - Frequent Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Hispanic or Latino | 1 Participants |
| Passive Control - December | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Hispanic or Latino | 3 Participants |
| Passive Control - December | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Unable to Obtain | 0 Participants |
| Passive Control - December | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Not Hispanic or Latino | 5 Participants |
| Active Control Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Hispanic or Latino | 1 Participants |
| Active Control Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Not Hispanic or Latino | 25 Participants |
| Active Control Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Unable to Obtain | 2 Participants |
| Ease Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Unable to Obtain | 0 Participants |
| Ease Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Hispanic or Latino | 0 Participants |
| Ease Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Not Hispanic or Latino | 36 Participants |
| Waiting for You Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Not Hispanic or Latino | 41 Participants |
| Waiting for You Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Hispanic or Latino | 2 Participants |
| Waiting for You Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Unable to Obtain | 1 Participants |
| Protect Yourself - Rare Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Not Hispanic or Latino | 19 Participants |
| Protect Yourself - Rare Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Hispanic or Latino | 1 Participants |
| Protect Yourself - Rare Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Unable to Obtain | 0 Participants |
| Protect Yourself - Frequent Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Unable to Obtain | 0 Participants |
| Protect Yourself - Frequent Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Hispanic or Latino | 2 Participants |
| Protect Yourself - Frequent Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Ethnicity | Not Hispanic or Latino | 24 Participants |
Other Pre-specified
Number of Patients Who Self-Scheduled a Flu Shot by Race
Patient self-schedules a flu vaccine through the patient portal This outcome measures whether the patient schedules an appointment within 4 weeks of the message send date, but the appointment can occur more than 4 weeks past the send date.
Time frame: Within 4 weeks of the message send date
Population: This outcome divides the population by race (as listed in the electronic health record)
| Arm | Measure | Group | Value (COUNT_OF_PARTICIPANTS) |
|---|---|---|---|
| Passive Control - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | American Indian or Alaska Native | 0 Participants |
| Passive Control - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Native Hawaiian or Other Pacific Islander | 0 Participants |
| Passive Control - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Unknown | 0 Participants |
| Passive Control - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Two or More | 0 Participants |
| Passive Control - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Asian | 2 Participants |
| Passive Control - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Black or African American | 1 Participants |
| Passive Control - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | White | 40 Participants |
| Active Control Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Two or More | 0 Participants |
| Active Control Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | American Indian or Alaska Native | 0 Participants |
| Active Control Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Black or African American | 4 Participants |
| Active Control Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | White | 57 Participants |
| Active Control Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Native Hawaiian or Other Pacific Islander | 0 Participants |
| Active Control Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Asian | 2 Participants |
| Active Control Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Unknown | 0 Participants |
| Ease Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Unknown | 0 Participants |
| Ease Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | American Indian or Alaska Native | 0 Participants |
| Ease Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | White | 57 Participants |
| Ease Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Asian | 1 Participants |
| Ease Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Black or African American | 4 Participants |
| Ease Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Two or More | 0 Participants |
| Ease Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Native Hawaiian or Other Pacific Islander | 0 Participants |
| Waiting for You Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Native Hawaiian or Other Pacific Islander | 1 Participants |
| Waiting for You Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Unknown | 0 Participants |
| Waiting for You Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | White | 71 Participants |
| Waiting for You Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Asian | 1 Participants |
| Waiting for You Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Two or More | 0 Participants |
| Waiting for You Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Black or African American | 1 Participants |
| Waiting for You Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | American Indian or Alaska Native | 0 Participants |
| Protect Yourself - Rare Message | Number of Patients Who Self-Scheduled a Flu Shot by Race | Black or African American | 0 Participants |
| Protect Yourself - Rare Message | Number of Patients Who Self-Scheduled a Flu Shot by Race | American Indian or Alaska Native | 0 Participants |
| Protect Yourself - Rare Message | Number of Patients Who Self-Scheduled a Flu Shot by Race | Asian | 0 Participants |
| Protect Yourself - Rare Message | Number of Patients Who Self-Scheduled a Flu Shot by Race | Native Hawaiian or Other Pacific Islander | 1 Participants |
| Protect Yourself - Rare Message | Number of Patients Who Self-Scheduled a Flu Shot by Race | White | 67 Participants |
| Protect Yourself - Rare Message | Number of Patients Who Self-Scheduled a Flu Shot by Race | Two or More | 0 Participants |
| Protect Yourself - Rare Message | Number of Patients Who Self-Scheduled a Flu Shot by Race | Unknown | 2 Participants |
| Protect Yourself - Frequent Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Two or More | 0 Participants |
| Protect Yourself - Frequent Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Black or African American | 1 Participants |
| Protect Yourself - Frequent Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Native Hawaiian or Other Pacific Islander | 0 Participants |
| Protect Yourself - Frequent Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Unknown | 2 Participants |
| Protect Yourself - Frequent Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | White | 50 Participants |
| Protect Yourself - Frequent Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | American Indian or Alaska Native | 0 Participants |
| Protect Yourself - Frequent Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Race | Asian | 0 Participants |
| Passive Control - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | White | 7 Participants |
| Passive Control - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | American Indian or Alaska Native | 0 Participants |
| Passive Control - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Two or More | 0 Participants |
| Passive Control - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Black or African American | 0 Participants |
| Passive Control - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Unknown | 1 Participants |
| Passive Control - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Asian | 0 Participants |
| Passive Control - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Native Hawaiian or Other Pacific Islander | 0 Participants |
| Active Control Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Black or African American | 1 Participants |
| Active Control Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Asian | 0 Participants |
| Active Control Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | White | 24 Participants |
| Active Control Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Native Hawaiian or Other Pacific Islander | 0 Participants |
| Active Control Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Two or More | 0 Participants |
| Active Control Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Unknown | 3 Participants |
| Active Control Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | American Indian or Alaska Native | 0 Participants |
| Ease Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | American Indian or Alaska Native | 0 Participants |
| Ease Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Two or More | 0 Participants |
| Ease Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Unknown | 1 Participants |
| Ease Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Asian | 0 Participants |
| Ease Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Black or African American | 1 Participants |
| Ease Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | White | 34 Participants |
| Ease Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Native Hawaiian or Other Pacific Islander | 0 Participants |
| Waiting for You Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Black or African American | 3 Participants |
| Waiting for You Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Asian | 1 Participants |
| Waiting for You Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Unknown | 0 Participants |
| Waiting for You Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | White | 40 Participants |
| Waiting for You Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | American Indian or Alaska Native | 0 Participants |
| Waiting for You Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Two or More | 0 Participants |
| Waiting for You Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Native Hawaiian or Other Pacific Islander | 0 Participants |
| Protect Yourself - Rare Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Two or More | 0 Participants |
| Protect Yourself - Rare Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | American Indian or Alaska Native | 0 Participants |
| Protect Yourself - Rare Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | White | 18 Participants |
| Protect Yourself - Rare Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Black or African American | 0 Participants |
| Protect Yourself - Rare Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Asian | 1 Participants |
| Protect Yourself - Rare Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Unknown | 1 Participants |
| Protect Yourself - Rare Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Native Hawaiian or Other Pacific Islander | 0 Participants |
| Protect Yourself - Frequent Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Black or African American | 1 Participants |
| Protect Yourself - Frequent Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Two or More | 0 Participants |
| Protect Yourself - Frequent Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Asian | 0 Participants |
| Protect Yourself - Frequent Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | White | 25 Participants |
| Protect Yourself - Frequent Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Native Hawaiian or Other Pacific Islander | 0 Participants |
| Protect Yourself - Frequent Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | American Indian or Alaska Native | 0 Participants |
| Protect Yourself - Frequent Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Race | Unknown | 0 Participants |
Other Pre-specified
Number of Patients Who Self-Scheduled a Flu Shot by Sex
Patient self-schedules a flu vaccine through the patient portal This outcome measures whether the patient schedules an appointment within 4 weeks of the message send date, but the appointment can occur more than 4 weeks past the send date.
Time frame: Within 4 weeks of the message send date
Population: This outcome divides the population by sex (as listed in the electronic health record)
| Arm | Measure | Group | Value (COUNT_OF_PARTICIPANTS) |
|---|---|---|---|
| Passive Control - November | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Female | 28 Participants |
| Passive Control - November | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Male | 15 Participants |
| Active Control Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Female | 38 Participants |
| Active Control Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Male | 25 Participants |
| Ease Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Female | 46 Participants |
| Ease Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Male | 16 Participants |
| Waiting for You Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Female | 45 Participants |
| Waiting for You Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Male | 29 Participants |
| Protect Yourself - Rare Message | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Male | 17 Participants |
| Protect Yourself - Rare Message | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Female | 53 Participants |
| Protect Yourself - Frequent Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Female | 31 Participants |
| Protect Yourself - Frequent Message - November | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Male | 22 Participants |
| Passive Control - December | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Male | 2 Participants |
| Passive Control - December | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Female | 6 Participants |
| Active Control Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Female | 13 Participants |
| Active Control Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Male | 15 Participants |
| Ease Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Female | 21 Participants |
| Ease Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Male | 15 Participants |
| Waiting for You Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Female | 25 Participants |
| Waiting for You Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Male | 19 Participants |
| Protect Yourself - Rare Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Male | 9 Participants |
| Protect Yourself - Rare Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Female | 11 Participants |
| Protect Yourself - Frequent Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Male | 17 Participants |
| Protect Yourself - Frequent Message - December | Number of Patients Who Self-Scheduled a Flu Shot by Sex | Female | 9 Participants |