Breast Cancer, Breast Diseases
Conditions
Brief summary
The BR-USCAD DS Module is a computer-assisted detection and diagnosis software based on a deep learning algorithm. This retrospective, fully-crossed, multi-reader, multi-case (MRMC) study aims to compare the performances of readers without and with the aid of the Breast Ultrasound Image Reviewed with Assistance of Computer-Assisted Detection and Diagnosis System (BR-USCAD DS) in interpreting breast ultrasound images of lesions.
Interventions
DIAGNOSTIC_TESTReader Group X - Session 1
Each rater in Reader Group X will interpret the dataset A cases in different random order without BR-USCAD DS and interpret the dataset B cases in different random order with BR-USCAD DS.
DIAGNOSTIC_TESTReader Group Y - Session 1
Each rater in Reader Group Y will interpret the dataset A cases in different random order with BR-USCAD DS and interpret the dataset B cases in different random order without BR-USCAD DS.
DIAGNOSTIC_TESTReader Group X - Session 2
Each rater in Reader Group X will interpret the dataset A cases in different random order with BR-USCAD DS and interpret the dataset B cases in different random order without BR-USCAD DS.
DIAGNOSTIC_TESTReader Group Y - Session 2
Each rater in Reader Group Y will interpret the dataset A cases in different random order without BR-USCAD DS and interpret the dataset B cases in different random order with BR-USCAD DS.
Sponsors
Columbia University
Taipei Veterans General Hospital, Taiwan
Virginia Polytechnic Institute and State University
TaiHao Medical Inc.
Study design
Allocation
RANDOMIZED
Intervention model
CROSSOVER
Primary purpose
DIAGNOSTIC
Masking
NONE
Eligibility
Sex/Gender
FEMALE
Age
21 Years to No maximum
Healthy volunteers
No
Inclusion criteria
* B-mode breast ultrasound image * Female, age 21 or older * Breast lesion images acquired before a biopsy or surgery - these images were retrospectively collected with histology report. * Non-biopsied benign lesions with negative follow-up for a minimum of 24 months * At least two orthogonal views of a lesion
Exclusion criteria
* Breast lesion images acquired after biopsy or surgery. * Any breast surgeries or interventional procedures in the 12 months prior to ultrasound imaging * Case demonstrating administrative or technical errors * Multiple lesions in one 2-D ultrasound image * Breast ultrasound images with Doppler, elastography, or other overlays present * Case with less than 2-year follow-up and without biopsy confirmation
Design outcomes
Primary
| Measure | Time frame | Description |
|---|---|---|
| Comparing the Area Under the LROC Curve | 10 weeks | The area under the LROC curve (AUC\_LROC) on the diagnosis of benign and malignant lesions was computed and compared for Aim 1 (baseline) and Arm 2 (with BU-CAD assistance) studies. |
Secondary
| Measure | Time frame | Description |
|---|---|---|
| The Reading Time Was Computed and Compared for Aim 1 (Baseline) and Arm 2 (With BU-CAD Assistance) Studies. | 10 weeks | Each reader's reading time of a case was automatically recorded by the BU-CAD system. the average reading times of 16 readers with and without outlier reading times were compared between the aided and unaided sessions using the paired t-test. |
| The Sensitivity, Specificity, PPV, and NPV Were Computed and Compared for Aim 1 (Baseline) and Arm 2 (With BU-CAD Assistance) Studies. | 10 weeks | The mean sensitivity, specificity, PPV, and NPV of 16 readers were calculated and compared between the aided and unaided sessions using McNemar's test. |
Countries
United States
Participant flow
Recruitment details
Six hundred and twenty-eight cases were randomized and split into two groups (A and B) with the same number of cases (314) in each group. Sixteen readers were randomized and split into two groups (X and Y) with the same number of readers (8) in each group.
Pre-assignment details
628 cases were randomly split into study datasets A and B. Each dataset had 314 cases. For each session, sub-datasets A and B were randomized within the group. Then, a reader read each sub-dataset either unaided or aided by the BU-CAD system.
Participants by arm
| Arm | Count |
|---|---|
| Without BU-CAD Decision Support Assistance Group X readers read group A cases(314) first unaided and then group B cases(314) aided. At least 5 weeks later, group X readers read group B cases(314) unaided, then group A cases(314) aided. | 8 |
| With BU-CAD Decision Support Assistance Group Y readers read group B cases(314) first unaided and then group A cases(314) aided. At least 5 weeks later, group Y readers read group A cases(314) unaided, then group B(314) cases aided. | 8 |
| Total | 16 |
Baseline characteristics
| Characteristic | Total | Without BU-CAD Decision Support Assistance | With BU-CAD Decision Support Assistance |
|---|---|---|---|
| Age, Categorical <=18 years | 0 Participants | 0 Participants | 0 Participants |
| Age, Categorical >=65 years | 0 Participants | 0 Participants | 0 Participants |
| Age, Categorical Between 18 and 65 years | 16 Participants | 8 Participants | 8 Participants |
| Race and Ethnicity Not Collected | 0 Participants | — | — |
| Region of Enrollment United States | 16 Participants | 8 Participants | 8 Participants |
| Sex/Gender, Customized Female | 11 Participants | 6 Participants | 5 Participants |
| Sex/Gender, Customized Male | 5 Participants | 2 Participants | 3 Participants |
Adverse events
| Event type | EG000 affected / at risk | EG001 affected / at risk | EG002 affected / at risk | EG003 affected / at risk |
|---|---|---|---|---|
| deaths Total, all-cause mortality | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 |
| other Total, other adverse events | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 |
| serious Total, serious adverse events | 0 / 0 | 0 / 0 | 0 / 0 | 0 / 0 |
Outcome results
Primary
Comparing the Area Under the LROC Curve
The area under the LROC curve (AUC\_LROC) on the diagnosis of benign and malignant lesions was computed and compared for Aim 1 (baseline) and Arm 2 (with BU-CAD assistance) studies.
Time frame: 10 weeks
Population: Of the study breast ultrasound cases, some contained malignant lesions and some contained breast contained benign lesions.
| Arm | Measure | Value (MEAN) |
|---|---|---|
| Arm 1 | Comparing the Area Under the LROC Curve | 0.7786 probability |
| Arm 2 | Comparing the Area Under the LROC Curve | 0.8160 probability |
p-value: <0.0595% CI: [0.019, 0.0557]OR-DBM model
Secondary
The Reading Time Was Computed and Compared for Aim 1 (Baseline) and Arm 2 (With BU-CAD Assistance) Studies.
Each reader's reading time of a case was automatically recorded by the BU-CAD system. the average reading times of 16 readers with and without outlier reading times were compared between the aided and unaided sessions using the paired t-test.
Time frame: 10 weeks
Population: Of the study breast ultrasound cases, some contained malignant lesions and some contained breast contained benign lesions.
| Arm | Measure | Value (MEAN) |
|---|---|---|
| Arm 1 | The Reading Time Was Computed and Compared for Aim 1 (Baseline) and Arm 2 (With BU-CAD Assistance) Studies. | 31.75 second |
| Arm 2 | The Reading Time Was Computed and Compared for Aim 1 (Baseline) and Arm 2 (With BU-CAD Assistance) Studies. | 18.96 second |
p-value: <0.05t-test, 2 sided
Secondary
The Sensitivity, Specificity, PPV, and NPV Were Computed and Compared for Aim 1 (Baseline) and Arm 2 (With BU-CAD Assistance) Studies.
The mean sensitivity, specificity, PPV, and NPV of 16 readers were calculated and compared between the aided and unaided sessions using McNemar's test.
Time frame: 10 weeks
Population: Of the study breast ultrasound cases, some contained malignant lesions and some contained breast contained benign lesions.
| Arm | Measure | Group | Value (MEAN) |
|---|---|---|---|
| Arm 1 | The Sensitivity, Specificity, PPV, and NPV Were Computed and Compared for Aim 1 (Baseline) and Arm 2 (With BU-CAD Assistance) Studies. | Sensitivity | 0.9225 proportion |
| Arm 1 | The Sensitivity, Specificity, PPV, and NPV Were Computed and Compared for Aim 1 (Baseline) and Arm 2 (With BU-CAD Assistance) Studies. | Specificity | 0.3165 proportion |
| Arm 1 | The Sensitivity, Specificity, PPV, and NPV Were Computed and Compared for Aim 1 (Baseline) and Arm 2 (With BU-CAD Assistance) Studies. | PPV | 0.4876 proportion |
| Arm 1 | The Sensitivity, Specificity, PPV, and NPV Were Computed and Compared for Aim 1 (Baseline) and Arm 2 (With BU-CAD Assistance) Studies. | NPV | 0.8623 proportion |
| Arm 2 | The Sensitivity, Specificity, PPV, and NPV Were Computed and Compared for Aim 1 (Baseline) and Arm 2 (With BU-CAD Assistance) Studies. | NPV | 0.8945 proportion |
| Arm 2 | The Sensitivity, Specificity, PPV, and NPV Were Computed and Compared for Aim 1 (Baseline) and Arm 2 (With BU-CAD Assistance) Studies. | Sensitivity | 0.9353 proportion |
| Arm 2 | The Sensitivity, Specificity, PPV, and NPV Were Computed and Compared for Aim 1 (Baseline) and Arm 2 (With BU-CAD Assistance) Studies. | PPV | 0.5056 proportion |
| Arm 2 | The Sensitivity, Specificity, PPV, and NPV Were Computed and Compared for Aim 1 (Baseline) and Arm 2 (With BU-CAD Assistance) Studies. | Specificity | 0.3611 proportion |
p-value: <0.0595% CI: [-0.2307, 0.2281]McNemar
p-value: <0.0595% CI: [0.0008, 0.2122]McNemar
p-value: <0.0595% CI: [-0.1824, 0.0104]McNemar
p-value: <0.0595% CI: [-0.0104, 0.1824]McNemar