Interpretable medical deep framework by logits-constraint attention guiding graph-based multi-scale fusion for Alzheimer’s disease analysis

• Published in: Pattern recognition Vol. 152; p. 110450
• Main Authors: Xu, Jinghao, Yuan, Chenxi, Ma, Xiaochuan, Shang, Huifang, Shi, Xiaoshuang, Zhu, Xiaofeng
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.08.2024
• Subjects: Alzheimer’s disease; Attention; Graph neural networks; Multi-scale feature fusion; Structural MRI; Structural MRI; Graph neural networks; Alzheimer’s disease; Multi-scale feature fusion; Attention
• DOI: 10.1016/j.patcog.2024.110450
• ISSN: 0031-3203

Deep learning using structural MRI has been widely applied to early diagnosis study of Alzheimer’s disease. Among existing methods, attention-based 3D subject-level methods can not only provide diagnosis results but also interpret the significant brain regions, thereby attracting considerable attention. However, the performance of previous attention-based methods might be still restricted by: (i) the gap between attention scores and semantic significant regions; (ii) using only single-scale features or simply fusing multi-scale information by addition or concatenation for classification decision-making. To overcome these two issues, we propose an innovative dual-branch model called LA-GMF, which consists of two major modules: logits-constraint attention (LA) and graph-based multi-scale fusion (GMF). The LA module is designed to guide the model to focus on key areas to enhance the diagnostic performance of local lesions, by reducing the inconsistency between attention scores and class prediction probabilities. Meanwhile, by combining the graph neural network and the self-attention mechanism, the GMF module not only introduces the interaction between patches, but also explores the correlation and complementarity between features at different scales, thereby extracting feature representations more comprehensively. Experiments on the popular ADNI and AIBL datasets validate the potential of our model in boosting early AD diagnosis accuracy. Additionally, our interpretation experiments demonstrate the superior interpretability performance of the proposed method over recent state-of-the-art attention-based methods. Our source codes are released at: https://github.com/nollexu/LA-GMF. [Display omitted] •We propose an interpretable dual-branch framework for Alzheimer’s disease analysis.•We design a logits-constraint attention to reduce the attention-semantic gap.•We design a graph-based multi-scale fusion module to fuse cross-scale information.