An Automatic Segmentation Method for Lesion Areas in Full-screen NBI Colorectal Endoscopy Using Deep Learning

• Published in: IEICE Transactions on Information and Systems Vol. E108.D; no. 9; pp. 1128 - 1137
• Main Authors: WU, Yongfei, KATAYAMA, Daisuke, KOIDE, Tetsushi, TAMAKI, Toru, YOSHIDA, Shigeto, MORIMOTO, Shin, OKA, Shiro, TANAKA, Shinji
• Format: Journal Article
• Language: English
• Published: The Institute of Electronics, Information and Communication Engineers 01.09.2025; 一般社団法人 電子情報通信学会
• Subjects: computer-aided-diagnosis (CAD); Endoscopy; Lesion Segmentation; Medical Deep Learning; NICE/JNET Classification
• ISSN: 0916-8532; 1745-1361
• DOI: 10.1587/transinf.2024EDP7283

In this paper, we propose an automatic segmentation method for detecting lesion areas from full-screen Narrow Band Imaging (NBI) endoscopic image frames using deep learning for real-time diagnosis support in endoscopy. In existing diagnosis support systems, doctors need to actively align lesion areas to accurately classify lesions. Therefore, we aim to develop a real-time diagnosis support system combining an automatic lesion segmentation algorithm, which can identify lesions in full-screen endoscopic image. We created a dataset of over 8000 images and verified the detection performance of multiple existing segmentation model structures. We realized that there is a serious problem of missing detection dealing with images with small lesion. We analyzed the possible reason and proposed a method of using convolutional backbone network for downsampling to retain effective information, and conducted experiments with a model structure using Dense Block and U-Net. The experimental results showed that the detection performance of our structure showed superiority over other models for small lesions. At the same time, CutMix, a data augmentation method added to the model learning method to further improve detection performance, was proven to be effective. The detection performance achieved an accuracy of 0.8603 ± 0.006 when evaluated using F-measure. In addition, our model showed the fastest processing speed in experimental test, which will be advantageous in the subsequent development of processing system for real-time clinical videos.