Advanced Cancer, Cancer, End of Life
Conditions
Keywords
Advanced Cancer, Cancer, Serious Illness, End of Life Care
Brief summary
The goal of this study is to test the accuracy of Large Language Model-generated serious illness communication (SIC) summaries, the feasibility of delivering the SIC summaries, and to collect perspectives on the SIC summaries from clinicians and participants with cancer. Large Language Models (LLMs) are artificial intelligence programs that can perform various natural language processing tasks.
Detailed description
The study will enroll up to 70 Dana-Farber patients admitted to Brigham and Women's hospital who have an elevated mortality prediction at admission. The research study procedures include randomization to intervention or control arms on admission (3:1 intervention to control). For patients in the intervention arm, a large language model query will summarize serious illness communication documentation in their medical record screening from the prior 6 months. A summary of the documentation will be sent to the inpatient and outpatient oncology clinicians within 24 hours of admission; these clinicians will be asked to review the summaries, discuss with the patient and incorporate into care plans as appropriate. For patients with no SIC documented on day 4 of the admission, a second email will be sent. Control patients will receive usual care, which is no email sent to their oncology teams. All patients will be offered to participate in an interview on day 4 of their admission about their communication with their care teams. For patients in both arms, inpatient and outpatient oncology attendings will be sent an email survey regarding SIC for the patient and, if applicable, the utility of the SIC summaries.
Interventions
OTHERLLM SIC Summary Email
A Large Language Model-based platform is integrated with the Health Vision Platform, a HIPAA-compliant, electronic health record data management system. Real-time summaries of prior SIC with be sent to clinicians caring for participants in the intervention arm. An email will be sent to the inpatient attending and responding clinician encouraging the team to review the SIC summary, discuss care preferences with the patient and ensure care is aligned with preferences and goals. If no SIC documentation is found, the email will instead encourage the outpatient oncologist to share information regarding undocumented SICs that may have occurred, as well as prompt inpatient and outpatient teams to engage the patient in SIC. For patients with no SIC documentation who are still admitted 72 hours later, the RA will send an additional email prompting SIC or asking the clinician to indicate that an SIC is not appropriate. Patients will also receive standard of care.
Sponsors
Dana-Farber Cancer Institute
Study design
Allocation
RANDOMIZED
Intervention model
PARALLEL
Primary purpose
OTHER
Masking
NONE
Eligibility
Sex/Gender
ALL
Age
18 Years to No maximum
Healthy volunteers
No
Inclusion criteria
Participant Inclusion Criteria: * Dana-Farber Cancer Institute participants * Admission to an inpatient solid tumor medical oncology service at BWH (including beds that are considered to be DFCI beds within BWH) * Age ≥ 18 years * Mortality prediction ≥ 40% * Admitted on a Sunday after 4pm, Monday or Tuesday before 4pm. These time restrictions are necessary for the workflow of this pragmatic pilot trial, as an individual will need to send out the intervention emails and interview patients approximately 48-72 hours after admission, both of which need to occur during the work week. Participant
Exclusion criteria
-Patients with elective inpatient admissions (typically for chemotherapy or other treatments) Clinician Inclusion Criteria: -The clinician fills one of the following roles for the enrolled patient: * inpatient attending on the medical oncology team * outpatient Dana-Farber medical oncologist
Design outcomes
Primary
| Measure | Time frame | Description |
|---|---|---|
| Serious Illness Conversation (SIC) summary accuracy | 1 week | Primary outcome: We will assess SIC summary accuracy by reviewing the source documentation for each bullet point of the 5 domains of the SIC summary. The RA will read each bullet point, read the source note for the date cited in the summary and indicate with a 1 (supported) or 0 (not supported) whether the documentation supports the summarized point. If a summarized point is scored as 0, the MPIs will independently review the item and assign a 1 or 0 to indicate whether the unsupported summary could negatively impact patient care. SIC summary accuracy will be determined at the patient level as the proportion of bullet points supported by underlying documentation divided by the total number of bullet points (i.e., a score from 0 to 1). We will determine the average SIC accuracy score across all patients for the study. We will also report the percent of LLM-SIC summary statements that are unsupported and clinically significant and provide qualitative descriptions of these instances. |
Secondary
| Measure | Time frame | Description |
|---|---|---|
| Feasibility (successful delivery of the intervention) | 1 week | Feasibility is defined as the successful delivery of the summary email, and will be calculated as the number of participants with successful delivery of the intervention email divided by the total number of intervention participants, with an 80% feasibility threshold. |
| Clinician Perspectives of Clinical Utility | 1 week | Clinician perspectives of clinical utility are defined as clinicians' views on the usefulness and relevance of serious illness communication (SIC) and, for intervention-arm clinicians, the LLM-generated SIC summaries for informing participant care. These perspectives will be assessed via structured REDCap surveys sent to both inpatient and outpatient clinicians, with follow-up phone interviews conducted by the research assistants for non-respondents. Clinician responses to surveys and interview questions will be summarized and stratified by intervention vs control groups. We will compare survey responses between the two study arms using Chisquared and Mann-Whitney U tests for binary and likert items, respectively. Responses to open-ended questions will be qualitatively summarized. |
| Participant Perspectives on Inpatient and Outpatient Clinician Understanding of their Care Preferences and Acceptability of SIC Summaries | 1 week | Participant acceptability is defined as participant perceptions of the appropriateness and usefulness of LLM-generated summaries of serious illness conversations (SICs). It will be assessed through structured in-person interviews including the validated "Heard and Understood" questionnaire and open-ended questions regarding the acceptability of SIC summaries and participant-clinician communication. Participant responses to interview questions will be summarized and stratified by intervention vs control groups. Responses to open-ended questions will be qualitatively summarized. |
| New Documentation of SIC | 1 week | New documentation of serious illness communication (SIC) during the index hospitalization is defined as any new note or entry created in the electronic health record (EHR) during the current hospital admission that reflects a discussion of the participant's values, goals, preferences, or prognosis, and meets the standardized codebook criteria for SIC. |
Countries
United States
Contacts
PRINCIPAL_INVESTIGATORCharlotta Lindvall, MD
Dana-Farber Cancer Institute
Outcome results
None listed