Vision transformer and CNN-based skin lesion analysis: classification of monkeypox

• Published in: Multimedia tools and applications Vol. 83; no. 28; pp. 71909 - 71923
• Main Author: Yolcu Oztel, Gozde
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.08.2024; Springer Nature B.V
• Subjects: Artificial neural networks; Classification; Computer Communication Networks; Computer Science; Data Structures and Information Theory; Deep learning; Ensemble learning; Image resolution; Lesions; Machine learning; Monkeypox; Multimedia Information Systems; Qualitative analysis; Special Purpose and Application-Based Systems; Track 2: Medical Applications of Multimedia; Ultrasonic testing; Deep learning; Vision transformer; Monkeypox; Transfer learning
• ISSN: 1573-7721; 1380-7501; 1573-7721
• DOI: 10.1007/s11042-024-19757-w

Monkeypox is an important health problem. Rapid diagnosis of monkeypox skin lesions and emergency isolation when necessary is essential. Also, some skin lesions, such as melanoma, can be fatal and must be rapidly distinguished. However, in some cases, it is difficult to distinguish the lesions visually. Methods such as dermoscopy, high-resolution ultrasound imaging, etc. can be used for better observation. But these methods are often based on qualitative analysis, subjective and time-consuming. Therefore, in this study, a quantitative and objective classification tool has been developed to assist dermatologists and scientists. The proposed system classifies seven skin lesions, including monkeypox. A popular approach Vision Transformer and some popular deep learning convolutional networks have been trained with the transfer learning approach and all results have been compared. Then, the models that show the best accuracy score have been combined to make the final prediction using bagging-ensemble learning. The proposed ensemble-based system produced 81.91% Accuracy, 65.94% Jaccard, 87.16% Precision, 74.12% Recall, and 78.16% Fscore values. In terms of different criteria metrics, the system produced competitive or even better results than the literature.