R-JaunLab: Automatic Multi-Class Recognition of Jaundice on Photos of Subjects with Region Annotation Networks

• Published in: Journal of digital imaging Vol. 34; no. 2; pp. 337 - 350
• Main Authors: Wang, Zheng, Xiao, Ying, Weng, Futian, Li, Xiaojun, Zhu, Danhua, Lu, Fanggen, Liu, Xiaowei, Hou, Muzhou, Meng, Yu
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 01.04.2021; Springer Nature B.V
• Subjects: Algorithms; Annotations; Artificial intelligence; Artificial neural networks; Bilirubin; Clinical medicine; Comparative studies; Digital imaging; Gallbladder; Gastroenterology; Globalization; Hepatitis; Hepatology; Hospitals; Humans; Imaging; Jaundice; Liver cancer; Localization; Medicine; Medicine & Public Health; Neural networks; Neural Networks, Computer; Original Paper; Pancreas; Pancreatic cancer; Physicians; Radiology; Recognition; Smartphones; Convolutional Neural Network; ); Region Annotation Network; Occult Jaundice; Total Serum Bilirubin; Region Annotation Network (RAN); Total Serum Bilirubin (TBil); Convolutional Neural Network (CNN)
• ISSN: 0897-1889; 1618-727X; 1618-727X
• DOI: 10.1007/s10278-021-00432-7

Jaundice occurs as a symptom of various diseases, such as hepatitis, the liver cancer, gallbladder or pancreas. Therefore, clinical measurement with special equipment is a common method that is used to identify the total serum bilirubin level in patients. Fully automated multi-class recognition of jaundice combines two key issues: (1) the critical difficulties in multi-class recognition of jaundice approaches contrasting with the binary class and (2) the subtle difficulties in multi-class recognition of jaundice represent extensive individuals variability of high-resolution photos of subjects, huge coherency between healthy controls and occult jaundice, as well as broadly inhomogeneous color distribution. We introduce a novel approach for multi-class recognition of jaundice to detect occult jaundice, obvious jaundice and healthy controls. First, region annotation network is developed and trained to propose eye candidates. Subsequently, an efficient jaundice recognizer is proposed to learn similarities, context, localization features and globalization characteristics on photos of subjects. Finally, both networks are unified by using shared convolutional layer. Evaluation of the structured model in a comparative study resulted in a significant performance boost (categorical accuracy for mean 91.38%) over the independent human observer. Our work was exceeded against the state-of-the-art convolutional neural network (96.85% and 90.06% for training and validation subset, respectively) and showed a remarkable categorical result for mean 95.33% on testing subset. The proposed network makes a performance better than physicians. This work demonstrates the strength of our proposal to help bringing an efficient tool for multi-class recognition of jaundice into clinical practice.