Toward automated severe pharyngitis detection with smartphone camera using deep learning networks

• Published in: Computers in biology and medicine Vol. 125; p. 103980
• Main Authors: Yoo, Tae Keun, Choi, Joon Yul, Jang, Younil, Oh, Ein, Ryu, Ik Hee
• Format: Journal Article
• Language: English
• Published: United States Elsevier Ltd 01.10.2020; Elsevier Limited
• Subjects: Automated diagnosis; Complications; Coronavirus Infections; Coronaviruses; COVID-19; Datasets; Deep Learning; Diagnosis; Disease control; Humans; Image Interpretation, Computer-Assisted - methods; Inflammation; Medical diagnosis; Medical imaging; Pandemics; Pharyngitis; Pharyngitis - diagnostic imaging; Pharynx; Pharynx - diagnostic imaging; Photography; Pneumonia, Viral; Signs and symptoms; Smartphone; Smartphones; Streptococcus infections; Telemedicine; Telemedicine - methods; Tonsillitis; Training; Transfer learning; Viral diseases; Tonsillitis; Deep learning; Pharyngitis; Telemedicine; Smartphone; Automated diagnosis
• ISSN: 0010-4825; 1879-0534
• DOI: 10.1016/j.compbiomed.2020.103980

Severe pharyngitis is frequently associated with inflammations caused by streptococcal pharyngitis, which can cause immune-mediated and post-infectious complications. The recent global pandemic of coronavirus disease (COVID-19) encourages the use of telemedicine for patients with respiratory symptoms. This study, therefore, purposes automated detection of severe pharyngitis using a deep learning framework with self-taken throat images. A dataset composed of two classes of 131 throat images with pharyngitis and 208 normal throat images was collected. Before the training classifier, we constructed a cycle consistency generative adversarial network (CycleGAN) to augment the training dataset. The ResNet50, Inception-v3, and MobileNet-v2 architectures were trained with transfer learning and validated using a randomly selected test dataset. The performance of the models was evaluated based on the accuracy and area under the receiver operating characteristic curve (ROC-AUC). The CycleGAN-based synthetic images reflected the pragmatic characteristic features of pharyngitis. Using the synthetic throat images, the deep learning model demonstrated a significant improvement in the accuracy of the pharyngitis diagnosis. ResNet50 with GAN-based augmentation showed the best ROC-AUC of 0.988 for pharyngitis detection in the test dataset. In the 4-fold cross-validation using the ResNet50, the highest detection accuracy and ROC-AUC achieved were 95.3% and 0.992, respectively. The deep learning model for smartphone-based pharyngitis screening allows fast identification of severe pharyngitis with a potential of the timely diagnosis of pharyngitis. In the recent pandemic of COVID-19, this framework will help patients with upper respiratory symptoms to improve convenience in diagnosis and reduce transmission. [Display omitted]