Quantitative and Qualitative Analysis of 18 Deep Convolutional Neural Network (CNN) Models with Transfer Learning to Diagnose COVID-19 on Chest X-Ray (CXR) Images

• Published in: SN computer science Vol. 4; no. 2; p. 141
• Main Authors: Chow, Li Sze, Tang, Goon Sheng, Solihin, Mahmud Iwan, Gowdh, Nadia Muhammad, Ramli, Norlisah, Rahmat, Kartini
• Format: Journal Article
• Language: English
• Published: Singapore Springer Nature Singapore 01.01.2023; Springer Nature B.V
• Subjects: Artificial neural networks; Chest; Computer Imaging; Computer Science; Computer Systems Organization and Communication Networks; Coronaviruses; COVID-19; Data Structures and Information Theory; Datasets; Decomposition; Diagnosis; Image classification; Information Systems and Communication Service; Learning; Medical imaging; Model accuracy; Original Research; Pattern Recognition and Graphics; Qualitative analysis; Respiratory diseases; Software Engineering/Programming and Operating Systems; Viral diseases; Vision; COVID-19; Convolutional neural networks (CNN); Transfer learning; Gradient-weighted class activation mapping (Grad-CAM)
• ISSN: 2661-8907; 2662-995X; 2661-8907
• DOI: 10.1007/s42979-022-01545-8

Coronavirus disease 2019 (COVID-19) is a disease caused by a novel strain of coronavirus, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), severely affecting the lungs. Our study aims to combine both quantitative and qualitative analysis of the convolutional neural network (CNN) model to diagnose COVID-19 on chest X-ray (CXR) images. We investigated 18 state-of-the-art CNN models with transfer learning, which include AlexNet, DarkNet-19, DarkNet-53, DenseNet-201, GoogLeNet, Inception-ResNet-v2, Inception-v3, MobileNet-v2, NasNet-Large, NasNet-Mobile, ResNet-18, ResNet-50, ResNet-101, ShuffleNet, SqueezeNet, VGG-16, VGG-19, and Xception. Their performances were evaluated quantitatively using six assessment metrics: specificity, sensitivity, precision, negative predictive value (NPV), accuracy, and F1-score. The top four models with accuracy higher than 90% are VGG-16, ResNet-101, VGG-19, and SqueezeNet. The accuracy of these top four models is between 90.7% and 94.3%; the F1-score is between 90.8% and 94.3%. The VGG-16 scored the highest accuracy of 94.3% and F1-score of 94.3%. The majority voting with all the 18 CNN models and top 4 models produced an accuracy of 93.0% and 94.0%, respectively. The top four and bottom three models were chosen for the qualitative analysis. A gradient-weighted class activation mapping (Grad-CAM) was used to visualize the significant region of activation for the decision-making of image classification. Two certified radiologists performed blinded subjective voting on the Grad-CAM images in comparison with their diagnosis. The qualitative analysis showed that SqueezeNet is the closest model to the diagnosis of two certified radiologists. It demonstrated a competitively good accuracy of 90.7% and F1-score of 90.8% with 111 times fewer parameters and 7.7 times faster than VGG-16. Therefore, this study recommends both VGG-16 and SqueezeNet as additional tools for the diagnosis of COVID-19.