The assessment of experimental artificial intelligence (AI) algorithms for the diagnosis of skin tumours against human performance

The condition observed is the diagnosis and management of skin tumours. The ground truth can take up to a year to be confirmed. Two groups of patients are recruited - 1. Those with baseline total body photographs taken 1-4 yrs prior to examination. Here, WHOLE BODY EXAMINATION occurs, and all discrete skin lesions are recruited with a > 4 mm longest diameter except for clinically non-suspicious amelanotic actinic keratoses and multiple lesions consistent with an inflammatory skin eruption or ephelides. In addition, all discrete lesions > 3 mm that are chosen for excision or monitoring by ground-truth assessment are recruited. 2. Patients undergoing routine excision or biopsy of pigmented skin lesions. Here, only these INDIVIDUAL LESIONS are examined. Clinicians with a variety of clinical expertise (dermatology residents/registrars, specialists) will be recruited with their level of expertise (years using dermoscopy, clinician classification) recorded. One novice clinician (dermatology resident/1st yr registrar) and one expert clinician (specialist in pigmented lesion clinics) will examine the patient. The diagnostic algorithms that will be COMPARED WITH THE CLINICIANS DIAGNOSIS/MANAGEMENT AND GROUND TRUTH are artificial intelligence-based that can be used on mobile phones. In our study, images are taken from the mobile phone of recruited patient lesions by the study researchers and analysed by cloud computing. The result is then returned to the researchers. Doctors and patients are not given the results. The output of these algorithms are Diagnosis (from 7 categories; 1. Melanoma 2. Melanocytic nevus 3. Pigmented Basal cell carcinoma 4. Pigmented Actinic keratosis / Bowen’s disease (intraepithelial carcinoma) 5. Benign (pigmented) keratotic lesion 6. Benign Vascular lesion 7. Dermatofibroma) and Management (dismiss, dermoscopy monitor or biopsy). However, the algorithms DO NOT influence patient management and are not revealed to the attending clinicians or patients. The following AI algorithms from MetaOptima will be used to generate the above results 1. 7-class classification * ISIC 2018 challenge winning model * An ensemble of Taxnet models (standard backbone network with an RNN) * An ensemble of Lesion Localization with Bias Unlearning models (LLBU) 2. Management decision * An ensemble of Taxnet models (standard backbone network with an RNN) to get ben/mal predictions and map to Dismiss | Monitor | Biopsy classes * An ensemble of Lesion Localization with Bias Unlearning models (LLBU) to get ben/mal predictions and map to Dismiss | Monitor | Biopsy classes The Active control = ground truth. Ground truth is determined by a descending hierarchy of: • Histopathology (ie. biopsy changed lesion). • Unchanged lesions on TBP = benign * Changed melanocytic lesions undergo 3-month dermoscopy monitoring (excise if change) • Subsequent unchanged 3-month digital dermoscopy OR insignificant changing long-term monitored lesions = benign • In vivo confocal microscopy (if available). • 2 of 2 independent readers viewing the dermoscopy images of changed lesions that are clinically benign (eg. Seborrheic keratoses, hemangiomas) = benign