Variability in human and automatic segmentation of melanocytic lesions

• Published in: 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society Vol. 2009; pp. 5789 - 5792
• Main Authors: Silletti, A., Peserico, E., Mantovan, A., Zattra, E., Peserico, A., Fortina, A.B.
• Format: Conference Proceeding Journal Article
• Language: English
• Published: United States IEEE 01.01.2009
• Subjects: Algorithms; Artificial Intelligence; Bibliographies; Dermoscopy - methods; Humans; Image analysis; Image Enhancement - methods; Image Interpretation, Computer-Assisted - methods; Image segmentation; Labeling; Lesions; Nevus, Pigmented - pathology; Noise level; Observer Variation; Pattern Recognition, Automated - methods; Reproducibility of Results; Sensitivity and Specificity; Skin; Upper bound; USA Councils
• ISSN: 1094-687X; 1557-170X; 1558-4615
• DOI: 10.1109/IEMBS.2009.5332543

In a double blind evaluation of 60 digital dermatoscopic images by 4 ldquojuniorrdquo, 4 ldquoseniorrdquo and 4 ldquoexpertrdquo dermatologists (dermatoscopy training respectively less than 1 year, between 1 and 5 years, and more than 5 years), a significant inter-operator variability was observed in melanocytic lesion border identification (with a disagreement of the order of 10 - 20% of the area of the lesions). Expert dermatologists showed greater agreement among themselves than with senior and junior dermatologists, and a slight tendency towards ldquotighterrdquo segmentations. The human inter-operator variability was then used to evaluate the segmentation accuracy of 4 algorithms, representative of the 3 fundamental state-of-the-art automated segmentation techniques and of a fourth, novel, technique. Our evaluation methodology addresses a number of crucial difficulties encountered in previous studies and may be of independent interest. 3 of the 4 algorithms showed considerably less agreement with expert dermatologists than even senior and junior dermatologists did (with a disagreement of the order of 30% of the area of the lesions); the remaining algorithm, however, showed agreement with expert dermatologists comparable to that of other expert dermatologists.