Echocardiographic Software, Echocardiography, Transthoracic
Conditions
Keywords
Echocardiography Interpretation
Brief summary
The goal of this blinded randomized controlled trial is to assess whether AI-assisted TTE interpretation can improve cardiologist efficiency and reduce interpretation variability while preserving accuracy compared to current methods in adult patients whose historical echocardiogram images were performed. The main questions it aims to answer is to: 1. Assess overall transthoracic echocardiogram (TTE) interpretation accuracy of an AI-assisted workflow compared to standard cardiologist-only and sonographer-assisted workflows to hypothesize if AI-assisted workflow for echocardiography interpretation is non-inferior in accuracy compared to standard workflows. 2. Evaluate cardiologist interpretation time using an AI-assisted workflow compared to standard cardiologist-only and sonographer-assisted workflows to determine if an AI-assisted workflow will result in a decrease in interpretation time compared to the cardiologist-only workflow. 3. To compare interpretation consistency and reliability compared to historical reports as well as consensus reports of all finalized cardiologist reports.
Interventions
OTHERAI-Assisted Echocardiogram Interpretation
Echocardiogram studies are pre-interpreted by the AI software and finalized by a blinded cardiologist.
OTHERSonographer-Assisted Echocardiogram Interpretation
(Standard of Care 1): Echocardiogram studies are pre-interpreted by a sonographer and finalized by a blinded cardiologist.
OTHERCardiologist-Only Echocardiogram Interpretation
(Standard of Care 2): Echocardiogram studies are interpreted solely by two cardiologists (one provides a preliminary interpretation another provides a blinded final interpretation).
Sponsors
Kaiser Permanente
Study design
Allocation
RANDOMIZED
Intervention model
PARALLEL
Primary purpose
DIAGNOSTIC
Masking
SINGLE (Subject)
Eligibility
Sex/Gender
ALL
Age
18 Years to No maximum
Healthy volunteers
No
Inclusion criteria
* ≥ 18 years of age * Comprehensive transthoracic echocardiogram study performed a KPNC Medical Center.
Exclusion criteria
* Known Congenital heart disease study * Known Ventricular assist device * Limited transthoracic echocardiogram study * Transesophageal echocardiogram study * Stress echocardiogram study
Design outcomes
Primary
| Measure | Time frame | Description |
|---|---|---|
| Accuracy of AI-Assisted Echocardiogram Interpretation Compared to Standard Workflows. | 6 months | The accuracy rate is defined as the proportion of transthoracic echocardiogram (TTE) studies for which the final cardiologist interpretation is not significantly different from the preliminary interpretation. We will assess the accuracy rate in the AI-assisted arm for non-inferiority versus the pooled accuracy rate across both the sonographer-assisted and cardiologist-only arms. The investigators will measure the proportion of TTE studies for which the final cardiologist report is substantially different from the preliminary report (e.g., the rate of substantial change). We define a TTE report to be substantially different from another TTE interpretation if any of its key elements are substantially different (LVEF, right ventriuclar function, chamber size, valvular regurgitation severity, valvular stenosis severity, and right ventricular systolic pressure). |
Secondary
| Measure | Time frame | Description |
|---|---|---|
| Secondary Measures of Accuracy (Superiority Analyses) | 6 months | Accuracy rates will also be analyzed for superiority rather than non-inferiority. The proportion of TTE studies for which the final cardiologist interpretation is not significantly different from the preliminary interpretation will be compared individually between the AI-assisted arm and each comparator arm (sonographer-assisted and cardiologist-only). 1. AI-prelim workflow vs. pooled sonographer-prelim and cardiologist-prelim workflow with a superiority test 2. AI-prelim workflow vs. sonographer-prelim workflow with a superiority test 3. AI-prelim workflow vs. cardiologist-prelim workflow with a superiority test |
| Accuracy Compared with Historical Cardiologist Interpretations | 6 months | Accuracy is defined as the proportion of TTE studies for which the final cardiologist report is substantially different from the historical report (e.g., the rate of substantial change). We define a TTE report to be substantially different from another TTE interpretation if any of its key elements are substantially different (LVEF, right ventriuclar function, chamber size, valvular regurgitation severity, valvular stenosis severity, and right ventricular systolic pressure). |
| Accuracy Compared with Consensus Cardiologist Interpretations | 6 months | Accuracy is defined as the proportion of TTE studies for which the consensus cardiologist report is substantially different from the preliminary report (e.g., the rate of substantial change). We define a TTE report to be substantially different from another TTE interpretation if any of its key elements are substantially different (LVEF, right ventriuclar function, chamber size, valvular regurgitation severity, valvular stenosis severity, and right ventricular systolic pressure). |
| Evaluation requiring two-grade change in valve disease severity as substantial change | 6 months | Accuracy is defined asthe proportion of TTE studies for which the final cardiologist report is substantially different from the preliminary report (e.g., the rate of substantial change). We define a TTE report to be substantially different from another TTE interpretation if any of its key elements are substantially different (LVEF, right ventricular function, chamber size, valvular regurgitation severity, valvular stenosis severity, and right ventricular systolic pressure). 1. AI-prelim workflow vs. pooled sonographer-prelim and cardiologist-prelim workflow with a superiority test 2. AI-prelim workflow vs. sonographer-prelim workflow with a superiority test 3. AI-prelim workflow vs. cardiologist-prelim workflow with a superiority test |
| Cardiologist Overread Time (Efficiency) | 6 months | To assess whether AI-assisted interpretation saves time, the investigators will compare the mean time in seconds required for the cardiologist to complete the overreading process across the three arms (i.e., TTE interpretation workflows) in our trial. Specifically, we will perform the following comparisons: 1. The mean cardiologist overread time in the AI-assisted arm versus the pooled results of the sonographer-assisted and cardiologist-only arms 2. The mean cardiologist overread time in the AI-assisted arm versus the sonographer-assisted arm 3. The mean cardiologist overread time in the AI-assisted arm versus the cardiologist-only arm |
| Comparison of Existing Standards of Care (Sonographer-Assisted vs Cardiologist-Only) | 6 months | The investigators will measure the proportion of TTE studies for which the final cardiologist report is substantially different from the preliminary report (e.g., the rate of substantial change) in sonographer-prelim and cardiologist-prelim arms. We define a TTE report to be substantially different from another TTE interpretation if any of its key elements are substantially different (LVEF, right ventricular function, chamber size, valvular regurgitation severity, valvular stenosis severity, and right ventricular systolic pressure). |
| Blinding | 6 months | In each of the arms, at the completion of each study for the final cardiologist assessment, the cardiologist will be asked to guess whether the preliminary assessment was AI or human. This will be used to calculate a blinding index comparing the AI-prelim arm vs. a composite of the two standard of care arms, as well as comparing the AI-prelim arm with the sonographer-prelim arm individually and comparing the AI-prelim arm with the cardiologist-prelim arm individually. |
Countries
United States
Contacts
PRINCIPAL_INVESTIGATORKeane K Lee
Kaiser Permanente
Outcome results
None listed