Melanoma
Conditions
Brief summary
This survey study purposes to determine and compare the biopsy/referral sensitivity and specificity of MelaFind to the average biopsy/referral sensitivity and specificity of dermatologists. 241 subjects logged into system but only 183 signed consents and completed the intake survey. Out of these 183, 155 were accounted for in the data analysis after exclusions were removed from the pool of subjects.
Detailed description
Early detection of melanoma is critical for favorable prognosis, since patients with earlier stage melanomas have a much higher probability of survival than with later stages. The traditional method of early detection has been with serial total body skin exams where the health care provider examines all skin surfaces, including mucosa, for suspicious pigmented lesions. Studies have demonstrated that the diagnostic accuracy of physicians for melanoma depends on the level of dermatological training. More important than being able to make a diagnosis of melanoma on clinical impression is the ability to make an appropriate decision to biopsy the lesion. Primary care physicians (PCPs) are often expected to screen for melanoma and only refer to dermatologists when there is a high clinical suspicion of melanoma. However, if PCPs are not adept at diagnosing melanoma, then opportunities for early diagnosis and treatment could be missed. Conversely, the morphology of benign pigmented lesions can often mimic that of early melanomas, resulting in potentially unnecessary dermatology referrals, biopsies, and patient anxiety. Studies have indicated that there is great variability in the ability of PCPs to make a correct decision to biopsy/refer a pigmented lesion (1.5 times greater than dermatologists) as well as for diagnosing melanoma (over 2.5 times greater than dermatologists). To aid in detection of early melanomas, new technologies are being developed. One such technology is MelaFind, an investigational device that has been developed to give a recommendation for biopsy (or not) of pigmented skin lesions to rule out melanoma. Our hypothesis is that MelaFind will have equal or better sensitivity than pigmented lesion experts in making an appropriate recommendation for biopsy (i.e., MelaFind will be at least as accurate as dermatologists in recommending biopsy for melanomas).
Interventions
None listed
Sponsors
MELA Sciences, Inc.
Study design
Observational model
COHORT
Time perspective
CROSS_SECTIONAL
Eligibility
Sex/Gender
ALL
Healthy volunteers
Yes
Inclusion criteria
* Board Certified physicians or equivalent
Exclusion criteria
* Did not participate in EOS Protocols 20061 or 20081
Design outcomes
Primary
| Measure | Time frame | Description |
|---|---|---|
| Comparison of Biopsy/Referral Sensitivity of MelaFind and Dermatologists (Pigmented Skin Lesion Experts and General Dermatologists) | April 2010 | Sensitivity is the proportion of positive cases (i.e., histologically confirmed melanoma) identified as positive. Specificity is the proportion of negative cases (i.e., histologically confirmed non-melanoma) identified as negative. Because the number of cases given to each dermatologist varied, both sensitivity and specificity were computed for each dermatologist. The primary outcome as stated was to compare the sensitivity and specificity of all dermatologists to that of MelaFind. These metrics, for both the dermatologists and MelaFind, were calculated based on the same 130 lesions. |
Secondary
| Measure | Time frame | Description |
|---|---|---|
| Comparison of Biopsy/Referral Sensitivity and Specificity of MelaFind to the Average of Biopsy/Referral Sensitivity & Specificity in Each of the Three Groups of Physicians: Pigmented Skin Lesion Experts, General Dermatologists, and Primary Care Physicians | December 2009 | Sensitivity is the proportion of positive cases (i.e., histologically confirmed melanoma) identified as positive. Specificity is the proportion of negative cases (i.e., histologically confirmed non-melanoma) identified as negative. Because the number of cases given to each dermatologist varied, both sensitivity and specificity were computed for each dermatologist. The primary outcome as stated was to compare the sensitivity and specificity of each group of physicians to that of Melafind, which is presented in the statistical analysis. |
| Determine the Interobserver Variability in Each of the Above Metrics Within Each of the Caregiver Groups. | December 2009 | Each physician was given up to 130 cases and asked whether or not they would biopsy the lesion. Interobserver variability was measured via the kappa statistic indicating how well the physicians' answers to that question agreed within each group. Kappa statistics are reported in the statistical analysis. while numbers rep, they dont reflect the sgreement among the subjects |
| To Compare Biopsy/Referral Performance and Diagnostic Performance Using Areas Under the Corresponding Receiver Operating Characteristic (ROC) Curves That Illustrate the Trade-offs Between Sensitivity and Specificity Between Three Groups of Physicians. | June 2010 | For each case reviewed, physicians were asked if they thought the lesion was a melanoma (diagnostic sensitivity/specificity) and whether or not they would biopsy or refer the lesion (biopsy/referral sensitivity/specificity. These measurements were compared using areas under the corresponding receiver operating characteristic curves. (see statistical analysis for results) ROC curves (reciver operating curves) are plotted on graphs with an x-axis of sensitivity and a y-axis of 1-specificity. |
Participant flow
Recruitment details
Physicians were selected from the membership list of the AAD and lists provided by SK&A Healthcare Information Solutions. A letter was then sent inviting them to participate. For each who replied positively,an access code was mailed with which to log on to the study. 241 subjects logged into system,183 signed consents & completed the Intake Survey.
Pre-assignment details
Participants were grouped according to responses to an Intake Survey, completed after Consent and before reading any study cases. A maximum of 130 cases were reviewed,each one consisting of 3 clinical images and a case history. MelaFind results were not provided as part of each case and were only used for sensitivity and specificity calculations.
Participants by arm
| Arm | Count |
|---|---|
| General Dermatologists Dermatologists were defined as board-certified dermatologists who did not participate in previous EOS Protocols 20061 and 20081 | 54 |
| Pigmented Skin Lesion Experts Pigmented Skin Lesion Expert were defined as board-certified dermatologists who spend at least 25% of their practice time evaluating pigmented skin lesions (PSLs) | 75 |
| Primary Care Physicians Primary Care Physicians(PCPs) were defined as physicians who deliver primary care service to adult patients (e.g., internists, general practitioners, family practitioners, and geriatricians). | 54 |
| Total | 183 |
Withdrawals & dropouts
| Period | Reason | FG000 | FG001 | FG002 |
|---|---|---|---|---|
| Overall Study | Less than 78 cases completed | 8 | 11 | 9 |
Baseline characteristics
| Characteristic | General Dermatologists | Pigmented Skin Lesion Experts | Primary Care Physicians | Total |
|---|---|---|---|---|
| Age, Categorical <=18 years | 0 Participants | 0 Participants | 0 Participants | 0 Participants |
| Age, Categorical >=65 years | 0 Participants | 0 Participants | 0 Participants | 0 Participants |
| Age, Categorical Between 18 and 65 years | 54 Participants | 75 Participants | 54 Participants | 183 Participants |
| Region of Enrollment United States | 54 participants | 75 participants | 54 participants | 183 participants |
| Sex/Gender, Customized Female | 16 participants | 36 participants | 23 participants | 75 participants |
| Sex/Gender, Customized Male | 32 participants | 35 participants | 26 participants | 93 participants |
| Sex/Gender, Customized Unknown | 6 participants | 4 participants | 5 participants | 15 participants |
Adverse events
| Event type | EG000 affected / at risk | EG001 affected / at risk | EG002 affected / at risk |
|---|---|---|---|
| deaths Total, all-cause mortality | — / — | — / — | — / — |
| other Total, other adverse events | 0 / 0 | 0 / 0 | 0 / 0 |
| serious Total, serious adverse events | 0 / 0 | 0 / 0 | 0 / 0 |
Outcome results
Primary
Comparison of Biopsy/Referral Sensitivity of MelaFind and Dermatologists (Pigmented Skin Lesion Experts and General Dermatologists)
Sensitivity is the proportion of positive cases (i.e., histologically confirmed melanoma) identified as positive. Specificity is the proportion of negative cases (i.e., histologically confirmed non-melanoma) identified as negative. Because the number of cases given to each dermatologist varied, both sensitivity and specificity were computed for each dermatologist. The primary outcome as stated was to compare the sensitivity and specificity of all dermatologists to that of MelaFind. These metrics, for both the dermatologists and MelaFind, were calculated based on the same 130 lesions.
Time frame: April 2010
Population: Participants were invited to enroll in this study via mail. Minimum number of participants was determined by statistician based on power analyses and number of cases completed. The time frame of the study was about 6 months and the comparison between Dermatologists and MelaFind is presented in the statistical analysis section below.
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Dermatologists | Comparison of Biopsy/Referral Sensitivity of MelaFind and Dermatologists (Pigmented Skin Lesion Experts and General Dermatologists) | Proportion of True Positive Cases (Sensitivity) | 0.72 Proportion of True Cases | 95% Confidence Interval 8.4 |
| Dermatologists | Comparison of Biopsy/Referral Sensitivity of MelaFind and Dermatologists (Pigmented Skin Lesion Experts and General Dermatologists) | Proportion of True Negative Cases (Specificity) | 0.51 Proportion of True Cases | 95% Confidence Interval 0 |
| MelaFind | Comparison of Biopsy/Referral Sensitivity of MelaFind and Dermatologists (Pigmented Skin Lesion Experts and General Dermatologists) | Proportion of True Positive Cases (Sensitivity) | 0.97 Proportion of True Cases | 95% Confidence Interval 0 |
| MelaFind | Comparison of Biopsy/Referral Sensitivity of MelaFind and Dermatologists (Pigmented Skin Lesion Experts and General Dermatologists) | Proportion of True Negative Cases (Specificity) | 0.09 Proportion of True Cases | 95% Confidence Interval 0 |
Comparison: Using True Positives/All Positives, sensitivity values were calculated for MelaFind as well as the average of the 110 dermatologists.p-value: <0.000195% CI: [0.18, 0.32]ANOVA
Secondary
Comparison of Biopsy/Referral Sensitivity and Specificity of MelaFind to the Average of Biopsy/Referral Sensitivity & Specificity in Each of the Three Groups of Physicians: Pigmented Skin Lesion Experts, General Dermatologists, and Primary Care Physicians
Sensitivity is the proportion of positive cases (i.e., histologically confirmed melanoma) identified as positive. Specificity is the proportion of negative cases (i.e., histologically confirmed non-melanoma) identified as negative. Because the number of cases given to each dermatologist varied, both sensitivity and specificity were computed for each dermatologist. The primary outcome as stated was to compare the sensitivity and specificity of each group of physicians to that of Melafind, which is presented in the statistical analysis.
Time frame: December 2009
Population: Each category was diminished after excluded subjects were taken into account. These included subjects who did not complete at least 78 cases, subjects who previously participated in other EOS studies, pediatricians, and a board eligible dermatologist.
| Arm | Measure | Group | Value (MEAN) | Dispersion |
|---|---|---|---|---|
| Dermatologists | Comparison of Biopsy/Referral Sensitivity and Specificity of MelaFind to the Average of Biopsy/Referral Sensitivity & Specificity in Each of the Three Groups of Physicians: Pigmented Skin Lesion Experts, General Dermatologists, and Primary Care Physicians | Proportion of True Positive Cases (Sensitivity) | 0.73 Proportion of True Cases | 95% Confidence Interval 8.7 |
| Dermatologists | Comparison of Biopsy/Referral Sensitivity and Specificity of MelaFind to the Average of Biopsy/Referral Sensitivity & Specificity in Each of the Three Groups of Physicians: Pigmented Skin Lesion Experts, General Dermatologists, and Primary Care Physicians | Proportion of True Negative Cases (Specificity) | 0.51 Proportion of True Cases | 95% Confidence Interval 0 |
| MelaFind | Comparison of Biopsy/Referral Sensitivity and Specificity of MelaFind to the Average of Biopsy/Referral Sensitivity & Specificity in Each of the Three Groups of Physicians: Pigmented Skin Lesion Experts, General Dermatologists, and Primary Care Physicians | Proportion of True Negative Cases (Specificity) | 0.50 Proportion of True Cases | 95% Confidence Interval 0 |
| MelaFind | Comparison of Biopsy/Referral Sensitivity and Specificity of MelaFind to the Average of Biopsy/Referral Sensitivity & Specificity in Each of the Three Groups of Physicians: Pigmented Skin Lesion Experts, General Dermatologists, and Primary Care Physicians | Proportion of True Positive Cases (Sensitivity) | 0.71 Proportion of True Cases | 95% Confidence Interval 8.1 |
| Primary Care Physicians | Comparison of Biopsy/Referral Sensitivity and Specificity of MelaFind to the Average of Biopsy/Referral Sensitivity & Specificity in Each of the Three Groups of Physicians: Pigmented Skin Lesion Experts, General Dermatologists, and Primary Care Physicians | Proportion of True Positive Cases (Sensitivity) | 0.71 Proportion of True Cases | 95% Confidence Interval 10.9 |
| Primary Care Physicians | Comparison of Biopsy/Referral Sensitivity and Specificity of MelaFind to the Average of Biopsy/Referral Sensitivity & Specificity in Each of the Three Groups of Physicians: Pigmented Skin Lesion Experts, General Dermatologists, and Primary Care Physicians | Proportion of True Negative Cases (Specificity) | 0.45 Proportion of True Cases | 95% Confidence Interval 0 |
| MelaFind | Comparison of Biopsy/Referral Sensitivity and Specificity of MelaFind to the Average of Biopsy/Referral Sensitivity & Specificity in Each of the Three Groups of Physicians: Pigmented Skin Lesion Experts, General Dermatologists, and Primary Care Physicians | Proportion of True Positive Cases (Sensitivity) | 0.97 Proportion of True Cases | 95% Confidence Interval 0 |
| MelaFind | Comparison of Biopsy/Referral Sensitivity and Specificity of MelaFind to the Average of Biopsy/Referral Sensitivity & Specificity in Each of the Three Groups of Physicians: Pigmented Skin Lesion Experts, General Dermatologists, and Primary Care Physicians | Proportion of True Negative Cases (Specificity) | 0.09 Proportion of True Cases | 95% Confidence Interval 0 |
Comparison: Sensitivityp-value: <0.000195% CI: [0.19, 0.34]ANOVA
Comparison: Sensitivityp-value: <0.000195% CI: [0.16, 0.31]ANOVA
Comparison: Sensitivityp-value: <0.000195% CI: [0.19, 0.33]ANOVA
Comparison: Specificityp-value: <0.000195% CI: [-0.46, -0.25]ANOVA
Comparison: Specificityp-value: <0.000195% CI: [-0.53, -0.31]ANOVA
Comparison: Specificityp-value: <0.000195% CI: [-0.51, -0.3]ANOVA
Secondary
Determine the Interobserver Variability in Each of the Above Metrics Within Each of the Caregiver Groups.
Each physician was given up to 130 cases and asked whether or not they would biopsy the lesion. Interobserver variability was measured via the kappa statistic indicating how well the physicians' answers to that question agreed within each group. Kappa statistics are reported in the statistical analysis. while numbers rep, they dont reflect the sgreement among the subjects
Time frame: December 2009
Population: The statistical analysis section contains the Kappa results within Each of the Caregiver Groups
| Arm | Measure | Value (NUMBER) |
|---|---|---|
| Dermatologists | Determine the Interobserver Variability in Each of the Above Metrics Within Each of the Caregiver Groups. | 5927 Number of Cases |
| MelaFind | Determine the Interobserver Variability in Each of the Above Metrics Within Each of the Caregiver Groups. | 8263 Number of Cases |
| Primary Care Physicians | Determine the Interobserver Variability in Each of the Above Metrics Within Each of the Caregiver Groups. | 5844 Number of Cases |
| MelaFind | Determine the Interobserver Variability in Each of the Above Metrics Within Each of the Caregiver Groups. | 20034 Number of Cases |
ANOVA
ANOVA
ANOVA
ANOVA
Secondary
To Compare Biopsy/Referral Performance and Diagnostic Performance Using Areas Under the Corresponding Receiver Operating Characteristic (ROC) Curves That Illustrate the Trade-offs Between Sensitivity and Specificity Between Three Groups of Physicians.
For each case reviewed, physicians were asked if they thought the lesion was a melanoma (diagnostic sensitivity/specificity) and whether or not they would biopsy or refer the lesion (biopsy/referral sensitivity/specificity. These measurements were compared using areas under the corresponding receiver operating characteristic curves. (see statistical analysis for results) ROC curves (reciver operating curves) are plotted on graphs with an x-axis of sensitivity and a y-axis of 1-specificity.
Time frame: June 2010
| Arm | Measure | Group | Value (GEOMETRIC_MEAN) | Dispersion |
|---|---|---|---|---|
| Dermatologists | To Compare Biopsy/Referral Performance and Diagnostic Performance Using Areas Under the Corresponding Receiver Operating Characteristic (ROC) Curves That Illustrate the Trade-offs Between Sensitivity and Specificity Between Three Groups of Physicians. | Area Under Curve for Biopsy/Referral | 0.65 Area Under Curve for biopsy/referral | Standard Deviation 0.03 |
| Dermatologists | To Compare Biopsy/Referral Performance and Diagnostic Performance Using Areas Under the Corresponding Receiver Operating Characteristic (ROC) Curves That Illustrate the Trade-offs Between Sensitivity and Specificity Between Three Groups of Physicians. | Area Under Curve for Diagnostic | 0.68 Area Under Curve for biopsy/referral | Standard Deviation 0.03 |
| MelaFind | To Compare Biopsy/Referral Performance and Diagnostic Performance Using Areas Under the Corresponding Receiver Operating Characteristic (ROC) Curves That Illustrate the Trade-offs Between Sensitivity and Specificity Between Three Groups of Physicians. | Area Under Curve for Biopsy/Referral | 0.63 Area Under Curve for biopsy/referral | Standard Deviation 0.03 |
| MelaFind | To Compare Biopsy/Referral Performance and Diagnostic Performance Using Areas Under the Corresponding Receiver Operating Characteristic (ROC) Curves That Illustrate the Trade-offs Between Sensitivity and Specificity Between Three Groups of Physicians. | Area Under Curve for Diagnostic | 0.66 Area Under Curve for biopsy/referral | Standard Deviation 0.03 |
| Primary Care Physicians | To Compare Biopsy/Referral Performance and Diagnostic Performance Using Areas Under the Corresponding Receiver Operating Characteristic (ROC) Curves That Illustrate the Trade-offs Between Sensitivity and Specificity Between Three Groups of Physicians. | Area Under Curve for Biopsy/Referral | 0.59 Area Under Curve for biopsy/referral | Standard Deviation 0.02 |
| Primary Care Physicians | To Compare Biopsy/Referral Performance and Diagnostic Performance Using Areas Under the Corresponding Receiver Operating Characteristic (ROC) Curves That Illustrate the Trade-offs Between Sensitivity and Specificity Between Three Groups of Physicians. | Area Under Curve for Diagnostic | 0.61 Area Under Curve for biopsy/referral | Standard Deviation 0.03 |