BTSC-TNAS: A neural architecture search-based transformer for brain tumor segmentation and classification

• Published in: Computerized medical imaging and graphics Vol. 110; p. 102307
• Main Authors: Liu, Xiao, Yao, Chong, Chen, Hongyi, Xiang, Rui, Wu, Hao, Du, Peng, Yu, Zekuan, Liu, Weifan, Liu, Jie, Geng, Daoying
• Format: Journal Article
• Language: English
• Published: United States Elsevier Ltd 01.12.2023
• Subjects: Brain; Brain Neoplasms - diagnostic imaging; Brain tumors multi-modal MR images; Classification; Diagnosis, Computer-Assisted; Humans; Image Processing, Computer-Assisted; Internal Medicine; Learning; Neural architecture search; Other; Segmentation; Transformer; Brain tumors multi-modal MR images; Transformer; Neural architecture search; Segmentation; Classification
• ISSN: 0895-6111; 1879-0771; 1879-0771
• DOI: 10.1016/j.compmedimag.2023.102307

Glioblastoma (GBM), isolated brain metastasis (SBM), and primary central nervous system lymphoma (PCNSL) possess a high level of similarity in histomorphology and clinical manifestations on multimodal MRI. Such similarities have led to challenges in the clinical diagnosis of these three malignant tumors. However, many existing models solely focus on either the task of segmentation or classification, which limits the application of computer-aided diagnosis in clinical diagnosis and treatment. To solve this problem, we propose a multi-task learning transformer with neural architecture search (NAS) for brain tumor segmentation and classification (BTSC-TNAS). In the segmentation stage, we use a nested transformer U-shape network (NTU-NAS) with NAS to directly predict brain tumor masks from multi-modal MRI images. In the tumor classification stage, we use the multiscale features obtained from the encoder of NTU-NAS as the input features of the classification network (MSC-NET), which are integrated and corrected by the classification feature correction enhancement (CFCE) block to improve the accuracy of classification. The proposed BTSC-TNAS achieves an average Dice coefficient of 80.86% and 87.12% for the segmentation of tumor region and the maximum abnormal region in clinical data respectively. The model achieves a classification accuracy of 0.941. The experiments performed on the BraTS 2019 dataset show that the proposed BTSC-TNAS has excellent generalizability and can provide support for some challenging tasks in the diagnosis and treatment of brain tumors. •A joint multi-task learning framework for the segmentation and classification of GBM, SBM, and PCNSL from multi-modal MRI.•NTU-NAS segmentation with a nested structure based on CNN and transformer.•NAS-TE blocks and NAS-Conv blocks are constructed to automatically search for the optimal network stacking without manual design.•Multi-scale feature-sharing and CFCE blocks are designed to correct and enhance tumor classification-related features.