Comparative Analysis of Vision Transformers and Conventional Convolutional Neural Networks in Detecting Referable Diabetic Retinopathy

• Published in: Ophthalmology science (Online) Vol. 4; no. 6; p. 100552
• Main Authors: Goh, Jocelyn Hui Lin, Ang, Elroy, Srinivasan, Sahana, Lei, Xiaofeng, Loh, Johnathan, Quek, Ten Cheer, Xue, Cancan, Xu, Xinxing, Liu, Yong, Cheng, Ching-Yu, Rajapakse, Jagath C., Tham, Yih-Chung
• Format: Journal Article
• Language: English
• Published: Netherlands Elsevier Inc 01.11.2024; Elsevier
• Subjects: Convolutional neural network; Original; Referable diabetic retinopathy; Retinal photographs; Vision transformer; CNN; VAN; Convolutional neural network; AI; Referable diabetic retinopathy; DR; SWIN; AUC; Vision transformer; Retinal photographs; SEED; VGG; ViTs
• ISSN: 2666-9145; 2666-9145
• DOI: 10.1016/j.xops.2024.100552

Vision transformers (ViTs) have shown promising performance in various classification tasks previously dominated by convolutional neural networks (CNNs). However, the performance of ViTs in referable diabetic retinopathy (DR) detection is relatively underexplored. In this study, using retinal photographs, we evaluated the comparative performances of ViTs and CNNs on detection of referable DR. Retrospective study. A total of 48 269 retinal images from the open-source Kaggle DR detection dataset, the Messidor-1 dataset and the Singapore Epidemiology of Eye Diseases (SEED) study were included. Using 41 614 retinal photographs from the Kaggle dataset, we developed 5 CNN (Visual Geometry Group 19, ResNet50, InceptionV3, DenseNet201, and EfficientNetV2S) and 4 ViTs models (VAN_small, CrossViT_small, ViT_small, and Hierarchical Vision transformer using Shifted Windows [SWIN]_tiny) for the detection of referable DR. We defined the presence of referable DR as eyes with moderate or worse DR. The comparative performance of all 9 models was evaluated in the Kaggle internal test dataset (with 1045 study eyes), and in 2 external test sets, the SEED study (5455 study eyes) and the Messidor-1 (1200 study eyes). Area under operating characteristics curve (AUC), specificity, and sensitivity. Among all models, the SWIN transformer displayed the highest AUC of 95.7% on the internal test set, significantly outperforming the CNN models (all P < 0.001). The same observation was confirmed in the external test sets, with the SWIN transformer achieving AUC of 97.3% in SEED and 96.3% in Messidor-1. When specificity level was fixed at 80% for the internal test, the SWIN transformer achieved the highest sensitivity of 94.4%, significantly better than all the CNN models (sensitivity levels ranging between 76.3% and 83.8%; all P < 0.001). This trend was also consistently observed in both external test sets. Our findings demonstrate that ViTs provide superior performance over CNNs in detecting referable DR from retinal photographs. These results point to the potential of utilizing ViT models to improve and optimize retinal photo-based deep learning for referable DR detection. Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.