Early Melanoma Diagnosis With Sequential Dermoscopic Images

• Published in: IEEE transactions on medical imaging Vol. 41; no. 3; pp. 633 - 646
• Main Authors: Yu, Zhen, Nguyen, Jennifer, Nguyen, Toan D., Kelly, John, Mclean, Catriona, Bonnington, Paul, Zhang, Lei, Mar, Victoria, Ge, Zongyuan
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.03.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Cancer; Convolutional neural networks; Dermatology; Dermoscopy - methods; Diagnosis; Diagnostic Imaging; Diagnostic systems; Early Diagnosis; early melanoma diagnosis; Feature extraction; Humans; Image recognition; Lesion alignment; Lesions; Malignancy; Medical diagnosis; Medical imaging; Melanoma; Melanoma - diagnostic imaging; sequential dermoscopic images; Skin; Skin diseases; Skin lesions; Skin Neoplasms - diagnostic imaging; spatio-temporal feature learning
• ISSN: 0278-0062; 1558-254X
• DOI: 10.1109/TMI.2021.3120091

Dermatologists often diagnose or rule out early melanoma by evaluating the follow-up dermoscopic images of skin lesions. However, existing algorithms for early melanoma diagnosis are developed using single time-point images of lesions. Ignoring the temporal, morphological changes of lesions can lead to misdiagnosis in borderline cases. In this study, we propose a framework for automated early melanoma diagnosis using sequential dermoscopic images. To this end, we construct our method in three steps. First, we align sequential dermoscopic images of skin lesions using estimated Euclidean transformations, extract the lesion growth region by computing image differences among the consecutive images, and then propose a spatio-temporal network to capture the dermoscopic changes from aligned lesion images and the corresponding difference images. Finally, we develop an early diagnosis module to compute probability scores of malignancy for lesion images over time. We collected 179 serial dermoscopic imaging data from 122 patients to verify our method. Extensive experiments show that the proposed model outperforms other commonly used sequence models. We also compared the diagnostic results of our model with those of seven experienced dermatologists and five registrars. Our model achieved higher diagnostic accuracy than clinicians (63.69% vs. 54.33%, respectively) and provided an earlier diagnosis of melanoma (60.7% vs. 32.7% of melanoma correctly diagnosed on the first follow-up images). These results demonstrate that our model can be used to identify melanocytic lesions that are at high-risk of malignant transformation earlier in the disease process and thereby redefine what is possible in the early detection of melanoma.