Modeling the survival of colorectal cancer patients based on colonoscopic features in a feature ensemble vision transformer

• Published in: Computerized medical imaging and graphics Vol. 107; p. 102242
• Main Authors: Lo, Chung-Ming, Yang, Yi-Wen, Lin, Jen-Kou, Lin, Tzu-Chen, Chen, Wei-Shone, Yang, Shung-Haur, Chang, Shih-Ching, Wang, Huann-Sheng, Lan, Yuan-Tzu, Lin, Hung-Hsin, Huang, Sheng-Chieh, Cheng, Hou-Hsuan, Jiang, Jeng-Kai, Lin, Chun-Chi
• Format: Journal Article
• Language: English
• Published: United States Elsevier Ltd 01.07.2023
• Subjects: Colon cancer; Colonoscopy; Prognosis; Vision transformer; Vision transformer; Colon cancer; Prognosis; Colonoscopy
• ISSN: 0895-6111; 1879-0771
• DOI: 10.1016/j.compmedimag.2023.102242

The prognosis of patients with colorectal cancer (CRC) mostly relies on the classic tumor node metastasis (TNM) staging classification. A more accurate and convenient prediction model would provide a better prognosis and assist in treatment. From May 2014 to December 2017, patients who underwent an operation for CRC were enrolled. The proposed feature ensemble vision transformer (FEViT) used ensemble classifiers to benefit the combinations of relevant colonoscopy features from the pretrained vision transformer and clinical features, including sex, age, family history of CRC, and tumor location, to establish the prognostic model. A total of 1729 colonoscopy images were enrolled in the current retrospective study. For the prediction of patient survival, FEViT achieved an accuracy of 94 % with an area under the receiver operating characteristic curve of 0.93, which was better than the TNM staging classification (90 %, 0.83) in the experiment. FEViT reduced the limited receptive field and gradient disappearance in the conventional convolutional neural network and was a relatively effective and efficient procedure. The promising accuracy of FEViT in modeling survival makes the prognosis of CRC patients more predictable and practical. •A firstly approach using vision transformer to model survival using colonoscopic features.•The image features can be automatically and efficiently generated without human intervention.•The feature ensemble vision transformer (FEViT) integrated imaging and clinical features to generate a prognostic model.•FEViT reached the accuracy of 94 % which was better than the TNM staging classification (90 %).