Appointment Reminders
Conditions
Keywords
Behavioral Economics, Mental Health, Veterans, Veterans Health, Primary Care, Appointments and Schedules
Brief summary
No-shows, or missed visits are a persistent problem in all health care systems. They contribute to worsened patient access, longer wait times, and inefficient use limited health care resources. The VA's no-show rate has shown no improvement in years, resulting in a staggering 9 million ambulatory no-shows in Fiscal Year (FY) 2015. Appointment reminders are an essential and proven element to addressing no-shows but major research gaps exist. Behavioral economics (BE) and allied fields offer key insights that are relevant to developing innovation in the field of appointment reminders. Adding nudges informed by concepts such as social norms, behavioral intentions, clear instructions, and potential negative consequences to the Veteran and others is a novel but evidence-based way to create enhanced appointment reminders. Seemingly small changes to appointment letters can create measurable shifts in appointment attendance and no-shows. Even more, these behavioral nudges can produce large benefits when taken to scale and compounded across a population. This project will address several aims, including: developing BE-informed messages to incorporate into enhanced appointment reminders; evaluating the effect of several versions of enhanced appointment reminders; and identifying potential barriers and facilitators to widespread implementation of enhanced appointment reminder messages.
Detailed description
Objectives: The overarching objective of this proposal is to test the effectiveness of simple and scalable enhancements to appointment reminders in reducing no-shows, and prepare for larger-scale implementation. Plan: No-shows, or missed visits are a persistent problem in all health care systems. They contribute to worsened patient access, longer wait times, and inefficient use limited health care resources. The VA's no-show rate has shown no improvement in years, resulting in a staggering 9 million ambulatory no-shows in FY2015. Appointment reminders are an essential and proven element to addressing no-shows but major research gaps exist. Behavioral economics (BE) and allied fields offer key insights that are relevant to developing innovation in the field of appointment reminders. Adding nudges informed by concepts such as social norms, behavioral intentions, clear instructions, and potential negative consequences to the Veteran and others is a novel but evidence-based way to create enhanced appointment reminders. Seemingly small changes to appointment letters can create measurable shifts in appointment attendance and no-shows. Even more, these behavioral nudges can produce large benefits when taken to scale and compounded across a population. The investigators will address the following aims: 1. Aim 1: Develop and iteratively refine BE-informed messages based on Veterans' perceptions, and incorporate them into enhanced appointment reminders. 2. Aim 2: Determine the effect of four versions of enhanced appointment reminders on measures relevant to treatment access, compared with usual reminders. 3. Aim 3: Evaluate differences in treatment effect associated with four versions of enhanced appointment reminders. 4. Aim 4: Characterize potential barriers and facilitators to widespread implementation of enhanced appointment reminder messages. Methods: The investigators will first determine the validity of a series of draft messages, which are informed by concepts and principles from BE, psychology, and related fields and will be included in the intervention; the investigators will also explore potential mechanisms of action, and measure any harms by assessing Veterans' perceptions of the messages (Aim 1). After intervention refinement, the investigators will conduct a cluster randomized controlled trial to test 4 interventions, consisting of variations in nudges included in appointment letters, and compare them to usual appointment letters (Aims 2 and 3). The trial will be conducted at VA Portland Health Care System (VAPORHCS) in primary care and mental health clinics. The investigators will evaluate changes in no-show rates, attendance and cancellation rates and appointment wait times. Finally, the investigators will conduct a qualitative assessment with key informants to inform potential barriers and facilitators to implementation of revised appointment reminders (Aim 4). Relevance to VA's Mission: This proposal directly addresses two of the six Health Services Research and Development Service (HSR&D) priority areas: access and mental and behavioral health. First, reducing no-shows is likely to improve efficiency of access to care and reduce wait time. Second, this proposal includes a focus on interventions in mental health settings, which have especially high no-show rates and may have more potential for improvement. Medical Subject Heading Terms: Economics, Behavioral; Mental Health; Healthcare Quality, Access, and Evaluation
Interventions
BEHAVIORALSocial Norms + Behavioral Instructions
A letter with two types of nudges. One points out the common behavior of attending appointments. And one provides clear, specific instructions for making appointment changes. The appointment reminder also includes usual care (basic appointment information on date, location, and phone number(s) for scheduling changes).
BEHAVIORALCaring + Consequences for Others + Behavioral Instructions
A letter with three types of nudges. One suggests that the institution cares about the patient. One highlights a potential negative consequence for others if the patient no-shows. And one provides clear, specific instructions for making appointment changes. The appointment reminder also includes usual care (basic appointment information on date, location, and phone number(s) for scheduling changes).
BEHAVIORALCaring + Consequences for Self + Behavioral Instructions
A letter with three types of nudges. One suggests that the institution cares about the patients. One highlights potential negative consequences for the patient if s/he no-shows. And one provides clear, specific instructions for making appointment changes. The appointment reminder also includes usual care (basic appointment information on date, location, and phone number(s) for scheduling changes).
BEHAVIORALCaring + Consequences for Others + Consequences for Self + Social Norms + Behavioral Instructions
A letter with all types of nudges combined. One suggests that the institution cares about the patients. One highlights a potential negative consequence for others if the patient no-shows. One highlights potential negative consequences for the patient if s/he no-shows. One points out the common behavior of attending appointments. And one provides clear, specific instructions for making appointment changes. The appointment reminder also includes usual care (basic appointment information on date, location, and phone number(s) for scheduling changes).
Sponsors
Iowa City VA Health Care System
VA Salt Lake City Health Care System
VA Office of Research and Development
Study design
Allocation
RANDOMIZED
Intervention model
PARALLEL
Primary purpose
HEALTH_SERVICES_RESEARCH
Masking
NONE
Masking description
The intervention arms were assigned letters and the statistician was blinded for analysis.
Intervention model description
The primary objective is to systematically develop and test the effectiveness of a series of changes to appointment letters in a cluster RCT with the goal of reducing no-shows and increasing access. First, the investigators will iteratively obtain feedback and refine messages in draft interventions (Aim 1). Then evaluate the effects of four enhanced appointment reminder interventions informed by BE and the Theory of Planned Behavior, and compare them to usual care and each other (Aims 2 and 3). Appointments will be randomly allocated to one of the four interventions or usual care at the provider-level, with providers blinded to allocation. The trial will be conducted at eight locations across VAPORHCS in both mental health and primary care. The investigators will also conduct a qualitative assessment with key informants (Aim 4) to provide data that will be useful for future implementation (if intervention is effective) or suggest modifications to the intervention (if ineffective).
Eligibility
Sex/Gender
ALL
Age
18 Years to No maximum
Healthy volunteers
Yes
Inclusion criteria
* All subjects will be a minimum of 18 years old. * All subjects will be patients with scheduled outpatient appointments (either primary care or mental health) in the VA Portland Health Care System. * Race and ethnicity will not be used in determining inclusion or exclusion of subjects.
Exclusion criteria
* None
Design outcomes
Primary
| Measure | Time frame | Description |
|---|---|---|
| No-Show Rate | 1 year | The percentage of total appointments that are classified as a no-show, relative to the total number of appointments scheduled. The numerator (no-shows) consists of appointments marked as a no-show and appointments canceled by the patient or clinic after the appointment time. The denominator (total appointments) consists of no-shows and completed appointments. For calculation of missed appointments, canceled appointments are not included in the number of units analyzed. The numerator is multiplied by 100 to arrive at a percentage. |
Secondary
| Measure | Time frame | Description |
|---|---|---|
| Cancellation Rate | 1 year | The percentage of scheduled appointments that are marked as canceled. |
Countries
United States
Participant flow
Participants by arm
| Arm | Count |
|---|---|
| Primary Care Appointments: Usual Care A Primary Care appointment letter that includes basic appointment information on date, location, and phone number(s) for scheduling changes. | 4,916 |
| Primary Care Appointments: Caring + Consequences for Self + Behavioral Instructions A Primary Care appointment letter with three types of nudges. One suggests that the institution cares about the patients. One highlights potential negative consequences for the patient if s/he no-shows. And one provides clear, specific instructions for making appointment changes. The appointment reminder also includes usual care (basic appointment information on date, location, and phone number(s) for scheduling changes). | 6,592 |
| Primary Care Appointments: Caring + Consequences for Others + Behavioral Instructions A Primary Care appointment letter with three types of nudges. One suggests that the institution cares about the patient. One highlights a potential negative consequence for others if the patient no-shows. And one provides clear, specific instructions for making appointment changes. The appointment reminder also includes usual care (basic appointment information on date, location, and phone number(s) for scheduling changes). | 5,759 |
| Primary Care Appointments: Social Norms + Behavioral Instructions A Primary Care appointment letter with two types of nudges. One points out the common behavior of attending appointments. And one provides clear, specific instructions for making appointment changes. The appointment reminder also includes usual care (basic appointment information on date, location, and phone number(s) for scheduling changes). | 4,353 |
| Primary Care Appointments: Combination of All Nudges A Primary Care appointment letter with all types of nudges combined. One suggests that the institution cares about the patients. One highlights a potential negative consequence for others if the patient no-shows. One highlights potential negative consequences for the patient if s/he no-shows. One points out the common behavior of attending appointments. And one provides clear, specific instructions for making appointment changes. The appointment reminder also includes usual care (basic appointment information on date, location, and phone number(s) for scheduling changes). | 5,920 |
| Mental Health Appointments: Usual Care A Mental Health appointment letter that includes basic appointment information on date, location, and phone number(s) for scheduling changes. | 1,641 |
| Mental Health Appointments: Caring + Consequences for Self + Behavioral Instructions A Mental Health appointment letter with three types of nudges. One suggests that the institution cares about the patients. One highlights potential negative consequences for the patient if s/he no-shows. And one provides clear, specific instructions for making appointment changes. The appointment reminder also includes usual care (basic appointment information on date, location, and phone number(s) for scheduling changes). | 1,666 |
| Mental Health Appointments: Caring + Consequences for Others + Behavioral Instructions A Mental Health appointment letter with three types of nudges. One suggests that the institution cares about the patient. One highlights a potential negative consequence for others if the patient no-shows. And one provides clear, specific instructions for making appointment changes. The appointment reminder also includes usual care (basic appointment information on date, location, and phone number(s) for scheduling changes). | 2,305 |
| Mental Health Appointments: Social Norms + Behavioral Instructions A Mental Health appointment letter with two types of nudges. One points out the common behavior of attending appointments. And one provides clear, specific instructions for making appointment changes. The appointment reminder also includes usual care (basic appointment information on date, location, and phone number(s) for scheduling changes). | 2,250 |
| Mental Health Appointments: Combination of All Nudges A Mental Health appointment letter with all types of nudges combined. One suggests that the institution cares about the patients. One highlights a potential negative consequence for others if the patient no-shows. One highlights potential negative consequences for the patient if s/he no-shows. One points out the common behavior of attending appointments. And one provides clear, specific instructions for making appointment changes. The appointment reminder also includes usual care (basic appointment information on date, location, and phone number(s) for scheduling changes). | 1,558 |
| Total | 36,960 |
Baseline characteristics
| Characteristic | Mental Health Appointments: Usual Care | Total | Mental Health Appointments: Combination of All Nudges | Mental Health Appointments: Social Norms + Behavioral Instructions | Mental Health Appointments: Caring + Consequences for Others + Behavioral Instructions | Mental Health Appointments: Caring + Consequences for Self + Behavioral Instructions | Primary Care Appointments: Caring + Consequences for Self + Behavioral Instructions | Primary Care Appointments: Combination of All Nudges | Primary Care Appointments: Social Norms + Behavioral Instructions | Primary Care Appointments: Caring + Consequences for Others + Behavioral Instructions | Primary Care Appointments: Usual Care |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Age, Continuous Mental Health | 50.06 years STANDARD_DEVIATION 15.95 | 49.99 years STANDARD_DEVIATION 16.01 | 49.29 years STANDARD_DEVIATION 15.75 | 49.37 years STANDARD_DEVIATION 15.85 | 51.39 years STANDARD_DEVIATION 16.16 | 49.49 years STANDARD_DEVIATION 16.22 | — | — | — | — | — |
| Age, Continuous Primary Care | — | 63.75 years STANDARD_DEVIATION 15.53 | — | — | — | — | 64.81 years STANDARD_DEVIATION 14.85 | 62.62 years STANDARD_DEVIATION 16.14 | 62.87 years STANDARD_DEVIATION 16.08 | 62.77 years STANDARD_DEVIATION 15.92 | 65.60 years STANDARD_DEVIATION 14.44 |
| Race (NIH/OMB) Mental Health American Indian or Alaska Native | 19 Participants | 147 Participants | 29 Participants | 37 Participants | 37 Participants | 25 Participants | — | — | — | — | — |
| Race (NIH/OMB) Mental Health Asian | 27 Participants | 140 Participants | 25 Participants | 34 Participants | 33 Participants | 21 Participants | — | — | — | — | — |
| Race (NIH/OMB) Mental Health Black or African American | 83 Participants | 436 Participants | 87 Participants | 101 Participants | 96 Participants | 69 Participants | — | — | — | — | — |
| Race (NIH/OMB) Mental Health More than one race | 33 Participants | 188 Participants | 36 Participants | 47 Participants | 42 Participants | 30 Participants | — | — | — | — | — |
| Race (NIH/OMB) Mental Health Native Hawaiian or Other Pacific Islander | 12 Participants | 98 Participants | 20 Participants | 25 Participants | 29 Participants | 12 Participants | — | — | — | — | — |
| Race (NIH/OMB) Mental Health Unknown or Not Reported | 216 Participants | 1171 Participants | 214 Participants | 276 Participants | 245 Participants | 220 Participants | — | — | — | — | — |
| Race (NIH/OMB) Mental Health White | 1251 Participants | 7240 Participants | 1147 Participants | 1730 Participants | 1823 Participants | 1289 Participants | — | — | — | — | — |
| Race (NIH/OMB) Primary Care American Indian or Alaska Native | — | 357 Participants | — | — | — | — | 91 Participants | 72 Participants | 58 Participants | 71 Participants | 65 Participants |
| Race (NIH/OMB) Primary Care Asian | — | 281 Participants | — | — | — | — | 60 Participants | 57 Participants | 70 Participants | 45 Participants | 49 Participants |
| Race (NIH/OMB) Primary Care Black or African American | — | 891 Participants | — | — | — | — | 218 Participants | 199 Participants | 147 Participants | 172 Participants | 155 Participants |
| Race (NIH/OMB) Primary Care More than one race | — | 307 Participants | — | — | — | — | 70 Participants | 63 Participants | 54 Participants | 59 Participants | 61 Participants |
| Race (NIH/OMB) Primary Care Native Hawaiian or Other Pacific Islander | — | 230 Participants | — | — | — | — | 46 Participants | 63 Participants | 45 Participants | 39 Participants | 37 Participants |
| Race (NIH/OMB) Primary Care Unknown or Not Reported | — | 3458 Participants | — | — | — | — | 767 Participants | 706 Participants | 491 Participants | 833 Participants | 661 Participants |
| Race (NIH/OMB) Primary Care White | — | 22016 Participants | — | — | — | — | 5340 Participants | 4760 Participants | 3488 Participants | 4540 Participants | 3888 Participants |
| Sex/Gender, Customized Mental Health - Female | 300 participants | 1848 participants | 306 participants | 362 participants | 477 participants | 403 participants | — | — | — | — | — |
| Sex/Gender, Customized Mental Health - Male | 1341 participants | 7572 participants | 1252 participants | 1888 participants | 1828 participants | 1263 participants | — | — | — | — | — |
| Sex/Gender, Customized Primary Care - Female | — | 2738 participants | — | — | — | — | 564 participants | 587 participants | 444 participants | 839 participants | 349 participants |
| Sex/Gender, Customized Primary Care - Male | — | 24757 participants | — | — | — | — | 6028 participants | 5333 participants | 3909 participants | 4920 participants | 4567 participants |
Adverse events
| Event type | EG000 affected / at risk | EG001 affected / at risk | EG002 affected / at risk | EG003 affected / at risk | EG004 affected / at risk |
|---|---|---|---|---|---|
| deaths Total, all-cause mortality | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 |
| other Total, other adverse events | 0 / 7,768 | 0 / 9,454 | 0 / 9,059 | 1 / 7,196 | 0 / 8,685 |
| serious Total, serious adverse events | 0 / 7,768 | 0 / 9,454 | 0 / 9,059 | 0 / 7,196 | 0 / 8,685 |
Outcome results
Primary
No-Show Rate
The percentage of total appointments that are classified as a no-show, relative to the total number of appointments scheduled. The numerator (no-shows) consists of appointments marked as a no-show and appointments canceled by the patient or clinic after the appointment time. The denominator (total appointments) consists of no-shows and completed appointments. For calculation of missed appointments, canceled appointments are not included in the number of units analyzed. The numerator is multiplied by 100 to arrive at a percentage.
Time frame: 1 year
Population: The number of participants analyzed is greater than listed in Participant Flow because these results include two additional arms listed for analysis: all 4 Primary Care intervention arms, and all 4 Mental Health intervention arms. These were not individual arms to which participants were randomized but rather a combination of arms for purposes of analysis.~The number of appointments analyzed is greater than the number of participants because some participants had multiple appointments.
| Arm | Measure | Value (NUMBER) |
|---|---|---|
| Primary Care Appointments: Usual Care | No-Show Rate | 11.9 percentage of appointments no showed |
| Primary Care Appointments: Caring + Consequences for Self + Behavioral Instructions | No-Show Rate | 11.1 percentage of appointments no showed |
| Primary Care Appointments: Caring + Consequences for Others + Behavioral Instructions | No-Show Rate | 10.5 percentage of appointments no showed |
| Primary Care Appointments: Social Norms + Behavioral Instructions | No-Show Rate | 12.1 percentage of appointments no showed |
| Primary Care Appointments: Combination of All Nudges | No-Show Rate | 10.6 percentage of appointments no showed |
| Primary Care - All 4 Primary Care Intervention Arms Combined | No-Show Rate | 11.1 percentage of appointments no showed |
| Mental Health Appointments: Usual Care | No-Show Rate | 18.0 percentage of appointments no showed |
| Mental Health Appointments: Caring + Consequences for Self + Behavioral Instructions | No-Show Rate | 19.2 percentage of appointments no showed |
| Mental Health Appointments: Caring + Consequences for Others + Behavioral Instructions | No-Show Rate | 21.9 percentage of appointments no showed |
| Mental Health Appointments: Social Norms + Behavioral Instructions | No-Show Rate | 20.0 percentage of appointments no showed |
| Mental Health Appointments: Combination of All Nudges | No-Show Rate | 18.9 percentage of appointments no showed |
| Mental Health - All 4 Mental Health Intervention Arms Combined | No-Show Rate | 20.1 percentage of appointments no showed |
p-value: 0.14595% CI: [0.956, 1.357]Regression, Logistic
p-value: 0.29895% CI: [0.912, 1.35]Regression, Logistic
p-value: 0.12795% CI: [0.955, 1.452]Regression, Logistic
p-value: 0.28695% CI: [0.911, 1.37]Regression, Logistic
p-value: 0.18395% CI: [0.925, 1.504]Regression, Logistic
p-value: 0.20995% CI: [0.903, 1.595]Regression, Logistic
p-value: 0.46395% CI: [0.799, 1.638]Regression, Logistic
p-value: 0.07495% CI: [0.972, 1.822]Regression, Logistic
p-value: 0.30695% CI: [0.867, 1.577]Regression, Logistic
p-value: 0.37395% CI: [0.845, 1.565]Regression, Logistic
Secondary
Cancellation Rate
The percentage of scheduled appointments that are marked as canceled.
Time frame: 1 year
Population: The number of participants analyzed is greater than listed in Participant Flow because these results include two additional arms listed for analysis: all 4 Primary Care intervention arms, and all 4 Mental Health intervention arms. These were not individual arms to which participants were randomized but rather a combination of arms for purposes of analysis.~The number of appointments analyzed is greater than the number of participants because some participants had multiple appointments.
| Arm | Measure | Value (NUMBER) |
|---|---|---|
| Primary Care Appointments: Usual Care | Cancellation Rate | 6.0 percentage of appointments cancelled |
| Primary Care Appointments: Caring + Consequences for Self + Behavioral Instructions | Cancellation Rate | 5.7 percentage of appointments cancelled |
| Primary Care Appointments: Caring + Consequences for Others + Behavioral Instructions | Cancellation Rate | 8.9 percentage of appointments cancelled |
| Primary Care Appointments: Social Norms + Behavioral Instructions | Cancellation Rate | 6.8 percentage of appointments cancelled |
| Primary Care Appointments: Combination of All Nudges | Cancellation Rate | 8.5 percentage of appointments cancelled |
| Primary Care - All 4 Primary Care Intervention Arms Combined | Cancellation Rate | 7.3 percentage of appointments cancelled |
| Mental Health Appointments: Usual Care | Cancellation Rate | 10.2 percentage of appointments cancelled |
| Mental Health Appointments: Caring + Consequences for Self + Behavioral Instructions | Cancellation Rate | 9.3 percentage of appointments cancelled |
| Mental Health Appointments: Caring + Consequences for Others + Behavioral Instructions | Cancellation Rate | 8.0 percentage of appointments cancelled |
| Mental Health Appointments: Social Norms + Behavioral Instructions | Cancellation Rate | 9.3 percentage of appointments cancelled |
| Mental Health Appointments: Combination of All Nudges | Cancellation Rate | 8.7 percentage of appointments cancelled |
| Mental Health - All 4 Mental Health Intervention Arms Combined | Cancellation Rate | 8.8 percentage of appointments cancelled |
p-value: 0.15395% CI: [0.826, 1.03]Regression, Logistic
p-value: 0.25295% CI: [0.777, 1.068]Regression, Logistic
p-value: 0.41395% CI: [0.815, 1.088]Regression, Logistic
p-value: 0.4195% CI: [0.83, 1.079]Regression, Logistic
p-value: 0.1195% CI: [0.766, 1.027]Regression, Logistic
p-value: 0.1495% CI: [0.654, 1.062]Regression, Logistic
p-value: 0.40495% CI: [0.657, 1.184]Regression, Logistic
p-value: 0.07295% CI: [0.538, 1.027]Regression, Logistic
p-value: 0.05795% CI: [0.6, 1.008]Regression, Logistic
p-value: 0.63795% CI: [0.697, 1.247]Regression, Logistic