Substance Abuse, Substance-Related Disorders, Substance Use
Conditions
Keywords
natural language processing, machine learning, artificial intelligence, clinical decision support, unhealthy alcohol use, opioid use disorder, illicit drug use
Brief summary
The investigators propose to develop an open-source, publicly available machine learning model that health systems could download and apply to their electronic health record data marts to screen for substance misuse in their patients. The investigators hypothesize that the natural language processing algorithm can provide a standardized and interoperable approach for an automated daily screen on all hospitalized patients and provide better implementation fidelity for screening, brief intervention, and referral to treatment.
Detailed description
In 2016, nearly 30% hospital discharges in the United States (US) had a major diagnostic category for a substance-use related condition. Substance misuse ranks second among principal diagnoses for unplanned 7-day hospital readmission rates. Despite the availability of Screening, Brief Intervention, and Referral to Treatment (SBIRT) interventions, substance misuse is not part of the admission routine and only a minority of patients are screened for substance misuse in the hospital setting. This is particularly problematic, since among hospitalized inpatients, the prevalence of substance misuse is estimated to be as high as 25%, greater than either the general population or outpatient setting. Practical screening methods tailored for the hospital setting are needed. In the advent of Meaningful Use in the electronic health record (EHR), efficiency for alcohol detection may be improved by leveraging data collected during usual care. Documentation of substance use is common and occurs in over 96% of provider admission notes, but their free text format renders them difficult to mine and analyze. Natural Language Processing (NLP) and machine learning are subfields of artificial intelligence (AI) that provide a solution to analyze text data in the EHR to identify substance misuse. Modern NLP has fused with machine learning, another sub-field of artificial intelligence focused on learning from data. In particular, the most powerful NLP methods rely on supervised learning, a type of machine learning that takes advantage of current reference standards to make predictions about unseen cases In the earlier version of an NLP and machine learning tool, the investigators successfully used data from clinical notes collected in the first 24 hours of hospital admission to reach a sensitivity and specificity above 70% for identifying alcohol misuse. With nearly 36 million hospital admissions in 2016, a substance misuse classifier has potential to impact millions. In this study, the aim is to prospectively implement a substance misuse classifier to examine its effectiveness against current practice of all hospitalized adult patients at a tertiary health system. The health system has a mature screening system to examine substance misuse classifier performance against current practice of questionnaire screening. The hypothesis is that the substance misuse classifier may provide a standardized, interoperable, and accurate approach to screen hospitalized patients. Successful implementation of the classifier in hospitalized patients is a step towards an automated and comprehensive universal screening system for substance misuse.
Interventions
Clinical notes collected in the first day of hospital admission during usual care as input to natural language processing and machine learning algorithm.
Sponsors
Study design
Masking description
No masking as the manual screen is already part of usual care and the automated screen will become usual care in the post-period of the pre-post design.
Intervention model description
Quasi-experimental design as an interrupted time series
Eligibility
Inclusion criteria
* Ages 18 years old to 89 years old * Inpatient status during hospitalization * Length of stay greater than 24 hours
Exclusion criteria
* Cannot participate in the usual care SBIRT intervention * Death or obtunded during first 24 hours of admission * Discharged against medical advice * Transferred from another acute care hospital * Transferred to another acute care hospital
Design outcomes
Primary
| Measure | Time frame | Description |
|---|---|---|
| Proportion of Patients That Had a Universal Screen Positive and Received SBIRT (Screening, Brief Intervention, or Referral to Treatment) | 24 months | The primary outcome is the proportion of patients who received SBIRT after a positive universal screen for being at risk for substance misuse. The design is an interrupted time-series prospective observational study. |
Secondary
| Measure | Time frame | Description |
|---|---|---|
| All-cause Re-hospitalizations Following 6-months From the Index Hospital Encounter | 12 months enrollment with 6 months follow-up for rehospitalization | We will compare healthcare utilization outcomes in all patients between pre- and post-periods controlling for all patient demographic and clinical characteristics. |
Countries
United States
Participant flow
Pre-assignment details
The comparison groups are pre-intervention and intervention. data from 31432 people were assessed prior to the intervention in 2022-2023 and data from 33564 people were assessed with the intervention from 2023-2024: data from 64996 people were assessed in total.
Participants by arm
| Arm | Count |
|---|---|
| Usual Care Before intervention | 31,432 |
| NLP (Natural Language Processing) Pre-screen: SMART-AI Automated processing of clinical notes collected during routine care in first 24 hours of hospital admission to identify individuals at-risk for substance misuse to receive standard-of-care full screening and assessment, brief intervention, or referral to treatment (SBIRT) intervention.
Processing of clinical notes in the EHR data collected during routine care: Clinical notes collected in the first day of hospital admission during usual care as input to natural language processing and machine learning algorithm. | 33,564 |
| Total | 64,996 |
Baseline characteristics
| Characteristic | Total | NLP (Natural Language Processing) Pre-screen: SMART-AI | Usual Care |
|---|---|---|---|
| Admission Type Elective | 28405 Participants | 14652 Participants | 13753 Participants |
| Admission Type Emergency | 36591 Participants | 18912 Participants | 17679 Participants |
| Age, Continuous | 56 years STANDARD_DEVIATION 19 | 56 years STANDARD_DEVIATION 19 | 56 years STANDARD_DEVIATION 19 |
| Discharge AMA | 797 Participants | 385 Participants | 412 Participants |
| Discharge Home | 44511 Participants | 23335 Participants | 21176 Participants |
| Discharge Home / Home Health | 11267 Participants | 5528 Participants | 5739 Participants |
| Discharge Hospice / Expired | 1932 Participants | 1004 Participants | 928 Participants |
| Discharge Long Term Acute Care | 336 Participants | 171 Participants | 165 Participants |
| Discharge Other Transfer | 260 Participants | 162 Participants | 98 Participants |
| Discharge Other / Unknown | 124 Participants | 74 Participants | 50 Participants |
| Discharge Psych | 197 Participants | 93 Participants | 104 Participants |
| Discharge Skilled Nursing Facility / Rehab | 5572 Participants | 2812 Participants | 2760 Participants |
| Elixhauser Comorbidity | 2.9 comorbidities STANDARD_DEVIATION 4.8 | 2.9 comorbidities STANDARD_DEVIATION 4.9 | 2.8 comorbidities STANDARD_DEVIATION 4.8 |
| Insurance Medicaid | 17442 Participants | 8714 Participants | 8728 Participants |
| Insurance Medicare | 27475 Participants | 14364 Participants | 13111 Participants |
| Insurance Other | 390 Participants | 186 Participants | 204 Participants |
| Insurance Private | 15476 Participants | 8038 Participants | 7438 Participants |
| Insurance Self Pay | 1097 Participants | 666 Participants | 431 Participants |
| Insurance Unknown | 3116 Participants | 1596 Participants | 1520 Participants |
| Length of Stay (LOS) | 5.0 days STANDARD_DEVIATION 6.5 | 4.9 days STANDARD_DEVIATION 6.4 | 5.1 days STANDARD_DEVIATION 6.5 |
| Patient Class Inpatient | 50223 Participants | 25992 Participants | 24231 Participants |
| Patient Class Observation | 14773 Participants | 7572 Participants | 7201 Participants |
| Race/Ethnicity, Customized Asian | 2111 Participants | 1064 Participants | 1047 Participants |
| Race/Ethnicity, Customized Black | 22789 Participants | 11650 Participants | 11139 Participants |
| Race/Ethnicity, Customized Hispanic | 15424 Participants | 8061 Participants | 7363 Participants |
| Race/Ethnicity, Customized Other | 2376 Participants | 1318 Participants | 1058 Participants |
| Race/Ethnicity, Customized Unknown | 675 Participants | 374 Participants | 301 Participants |
| Race/Ethnicity, Customized White | 21621 Participants | 11097 Participants | 10524 Participants |
| Region of Enrollment United States | 64996 participants | 33564 participants | 31432 participants |
| Sex: Female, Male Female | 37879 Participants | 19421 Participants | 18458 Participants |
| Sex: Female, Male Male | 27117 Participants | 14143 Participants | 12974 Participants |
Adverse events
| Event type | EG000 affected / at risk | EG001 affected / at risk |
|---|---|---|
| deaths Total, all-cause mortality | 0 / 0 | 0 / 0 |
| other Total, other adverse events | 0 / 0 | 0 / 0 |
| serious Total, serious adverse events | 0 / 0 | 0 / 0 |
Outcome results
Proportion of Patients That Had a Universal Screen Positive and Received SBIRT (Screening, Brief Intervention, or Referral to Treatment)
The primary outcome is the proportion of patients who received SBIRT after a positive universal screen for being at risk for substance misuse. The design is an interrupted time-series prospective observational study.
Time frame: 24 months
| Arm | Measure | Value (COUNT_OF_PARTICIPANTS) |
|---|---|---|
| Usual Care | Proportion of Patients That Had a Universal Screen Positive and Received SBIRT (Screening, Brief Intervention, or Referral to Treatment) | 1189 Participants |
| SMART-AI | Proportion of Patients That Had a Universal Screen Positive and Received SBIRT (Screening, Brief Intervention, or Referral to Treatment) | 1144 Participants |
All-cause Re-hospitalizations Following 6-months From the Index Hospital Encounter
We will compare healthcare utilization outcomes in all patients between pre- and post-periods controlling for all patient demographic and clinical characteristics.
Time frame: 12 months enrollment with 6 months follow-up for rehospitalization
| Arm | Measure | Value (COUNT_OF_PARTICIPANTS) |
|---|---|---|
| Usual Care | All-cause Re-hospitalizations Following 6-months From the Index Hospital Encounter | 9584 Participants |
| SMART-AI | All-cause Re-hospitalizations Following 6-months From the Index Hospital Encounter | 10241 Participants |