Skin Lesions
Conditions
Keywords
Dermatology, AI, Artificial Intelligence, Uncertainty Calibration, Dermoscopy, Bayesian Deep Learning, Human-AI Decision Model
Brief summary
The goal of this randomized controlled study is to compare the effect of a new, personalized uncertainty-aware decision model (FDM) to a standard image recognition model in improving the diagnostic accuracy while reducing diagnostic uncertainty in experienced dermatologists tasked with differentiating between melanomas, moles and other benign skin lesions. The main question it aims to answer: Is the FDM a feasible method for an improved human AI partnership in which trust is build, misdiagnoses are avoided, and uncertainty is duly introduced or reduced. The investigators expect to see only a slight increase in collective diagnostic accuracy for both interventions as the the human participants are skilled dermatologist and thus have high accuracies pre-intervention. The investigators expect to see a higher increase in diagnostic certainty for the FDM intervention compared to the diagnostic certainty in the Base Model intervention. The investigators expect to see a higher amount of diagnosis changes from incorrect to correct in the FDM group compared to the Base Model group. The investigators do not expect any learning effect during the study. Participants will start by answering a series of training cases consisting of images of skin lesions. These are used to train their individual FDM (only for the FDM-intervention group). From here, the participants will be randomized into two arms determining which of the two interventions they are exposed to. The participants will solve each case withouth any intervention first, and this reply will act as a control.
Detailed description
A detailed description of the FDM is presented in the references.
Interventions
OTHERBase Model
See arm description.
OTHERFDM
See arm description
Sponsors
Copenhagen Academy for Medical Education and Simulation
Technical University of Denmark
Study design
Allocation
RANDOMIZED
Intervention model
PARALLEL
Primary purpose
DIAGNOSTIC
Masking
NONE
Intervention model description
Each participant will undergo the same training phase. Subsequently, the participants are randomized into one of the intervention arms. Each participant will be their own control.
Eligibility
Sex/Gender
ALL
Healthy volunteers
Yes
Inclusion criteria
* Board certified dermatologists with clinical experience in dermoscopic diagnosis.
Exclusion criteria
* Doctors who have not yet finished their specialization and dermatologists. * Dermatologists without clinical experience in dermoscopic diagnosis.
Design outcomes
Primary
| Measure | Time frame | Description |
|---|---|---|
| Accuracy | Immediately after the intervention. | Diagnostic accuracy in differentiating between melanoma, nevus, and benign keratosis. Defined as the percentage of correct diagnoses. Ground truth is based on histopathologically verified diagnoses. |
Secondary
| Measure | Time frame | Description |
|---|---|---|
| Uncertainty | Immediately after the intervention. | Changes in self-assesed uncertainty ranging from 0 (very uncertain) to 10 (very certain) from pre- to post-intervention. |
| Cut-off uncertainty | Immediately after the intervention. | The self-assessed uncertainty of cases where the participant has clicked a "would you like to discuss this case with a collegue"-button. |
Contacts
CONTACTJulie Renata Bjerremand
STUDY_CHAIRMartin Tolsgaard, Professor
Copenhagen Academy for Medical Education and Simulation
Outcome results
None listed