Multimodal predictive classification of Alzheimer's disease based on attention‐combined fusion network: Integrated neuroimaging modalities and medical examination data

• Published in: IET image processing Vol. 17; no. 11; pp. 3153 - 3164
• Main Authors: Chen, Hui, Guo, Huiru, Xing, Longqiang, Chen, Da, Yuan, Ting, Zhang, Yunpeng, Zhang, Xuedian
• Format: Journal Article
• Language: English
• Published: Wiley 01.09.2023
• Subjects: Alzheimer's disease; attention mechanism; diagnosis; multi‐modal; prediction
• ISSN: 1751-9659; 1751-9667
• DOI: 10.1049/ipr2.12841

Early diagnosis of Alzheimer's disease (AD) plays a key role in preventing and responding to this neurodegenerative disease. It has shown that, compared with a single imaging modality‐based classification of AD, synergy exploration among multimodal neuroimages is beneficial for the pathological identification. However, effectively exploiting multimodal information is still a big challenge due to the lack of efficient fusion methods. Herein, a multimodal fusion network based on attention mechanism is proposed, in which magnetic resonance imaging (MRI) and positron emission computed tomography (PET) images are converted into feature vectors with the same dimension, while the demographic information and clinical data are preprocessed and converted into feature vectors through embedding. This attention model can focus on important feature points, fuse the multimodal information more effectively, and thus provide accurate diagnosis and prediction for different pathological stages. The results show that the model achieves an accuracy of 84.1% for triple classification tasks in normal cognition (NC) versus mild cognitive impairment (MCI) versus AD and 93.9% prediction accuracy in stable MCI (sMCI) versus progressive MCI (pMCI). In contrast to the existing multimodal diagnosis methods, our model yields a state‐of‐the‐art accuracy of AD diagnosis, which is powerful and promising in clinical practice. Better integration of features from different modalities using attentional mechanisms for comprehensive diagnosis of Alzheimer's disease by combining features from different modalities.