Dual resolution deep learning network with self-attention mechanism for classification and localisation of colorectal cancer in histopathological images

• Published in: Journal of clinical pathology Vol. 76; no. 8; pp. 524 - 530
• Main Authors: Xu, Yan, Jiang, Liwen, Huang, Shuting, Liu, Zhenyu, Zhang, Jiangyu
• Format: Journal Article
• Language: English
• Published: England BMJ Publishing Group Ltd and Association of Clinical Pathologists 01.08.2023; BMJ Publishing Group LTD
• Subjects: Classification; Colorectal cancer; Colorectal Neoplasms; computer-aided design; Datasets; Deep Learning; Diagnosis, Computer-Assisted - methods; Humans; image processing, computer-assisted; Morphology; Neural networks; Original research; Pathologists; Sensitivity and Specificity; Stains & staining; computer-aided design; image processing, computer-assisted; colorectal cancer
• ISSN: 0021-9746; 1472-4146
• DOI: 10.1136/jclinpath-2021-208042

AimsMicroscopic examination is a basic diagnostic technology for colorectal cancer (CRC), but it is very laborious. We developed a dual resolution deep learning network with self-attention mechanism (DRSANet) which combines context and details for CRC binary classification and localisation in whole slide images (WSIs), and as a computer-aided diagnosis (CAD) to improve the sensitivity and specificity of doctors’ diagnosis.MethodsRepresentative regions of interest (ROI) of each tissue type were manually delineated in WSIs by pathologists. Based on the same coordinates of centre position, patches were extracted at different magnification levels from the ROI. Specifically, patches from low magnification level contain contextual information, while from high magnification level provide important details. A dual-inputs network was designed to learn context and details simultaneously, and self-attention mechanism was used to selectively learn different positions in the images to enhance the performance.ResultsIn classification task, DRSANet outperformed the benchmark networks which only depended on the high magnification patches on two test set. Furthermore, in localisation task, DRSANet demonstrated a better localisation capability of tumour area in WSI with less areas of misidentification.ConclusionsWe compared DRSANet with benchmark networks which only use the patches from high magnification level. Experimental results reveal that the performance of DRSANet is better than the benchmark networks. Both context and details should be considered in deep learning method.