Uncertainty-inspired open set learning for retinal anomaly identification

• Published in: Nature communications Vol. 14; no. 1; p. 6757
• Main Authors: Wang, Meng, Lin, Tian, Wang, Lianyu, Lin, Aidi, Zou, Ke, Xu, Xinxing, Zhou, Yi, Peng, Yuanyuan, Meng, Qingquan, Qian, Yiming, Deng, Guoyao, Wu, Zhiqun, Chen, Junhong, Lin, Jianhong, Zhang, Mingzhi, Zhu, Weifang, Zhang, Changqing, Zhang, Daoqiang, Goh, Rick Siow Mong, Liu, Yong, Pang, Chi Pui, Chen, Xinjian, Chen, Haoyu, Fu, Huazhu
• Format: Journal Article
• Language: English
• Published: London Nature Publishing Group UK 24.10.2023; Nature Publishing Group; Nature Portfolio
• Subjects: 639/166/985; 639/166/987; 692/53/2421; 692/699/3161/3175; Aeronautics; Anomalies; Artificial intelligence; Categories; Classification; Computer science; Datasets; Deep learning; Diabetes; Diabetic retinopathy; Disease; Hospitals; Humanities and Social Sciences; Image quality; Learning; Medical personnel; Medical screening; multidisciplinary; Probability; Recognition; Retina; Science; Science (multidisciplinary); Uncertainty; Visual impairment
• ISSN: 2041-1723; 2041-1723
• DOI: 10.1038/s41467-023-42444-7

Failure to recognize samples from the classes unseen during training is a major limitation of artificial intelligence in the real-world implementation for recognition and classification of retinal anomalies. We establish an uncertainty-inspired open set (UIOS) model, which is trained with fundus images of 9 retinal conditions. Besides assessing the probability of each category, UIOS also calculates an uncertainty score to express its confidence. Our UIOS model with thresholding strategy achieves an F1 score of 99.55%, 97.01% and 91.91% for the internal testing set, external target categories (TC)-JSIEC dataset and TC-unseen testing set, respectively, compared to the F1 score of 92.20%, 80.69% and 64.74% by the standard AI model. Furthermore, UIOS correctly predicts high uncertainty scores, which would prompt the need for a manual check in the datasets of non-target categories retinal diseases, low-quality fundus images, and non-fundus images. UIOS provides a robust method for real-world screening of retinal anomalies. Failure to recognize samples from unseen classes is a major limitation of AI recognition and classification of retinal anomalies. Here, the authors present the Uncertainty-inspired Open Set learning model that categorises fundus images into pre-trained categories, and provides an uncertainty score that alerts the need for manual inspection when dealing with out-of-distribution images.