Multi-task joint learning network based on adaptive patch pruning for Alzheimer’s disease diagnosis and clinical score prediction

• Published in: Biomedical signal processing and control Vol. 95; p. 106398
• Main Authors: Liu, Fangyu, Yuan, Shizhong, Li, Weimin, Xu, Qun, Wu, Xing, Han, Ke, Wang, Jingchao, Miao, Shang
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.09.2024
• Subjects: Adaptive patch pruning; Alzheimer’s disease; Multi-modal fusion; Multi-task learning; Weakly supervised learning; Alzheimer’s disease; Adaptive patch pruning; Multi-modal fusion; Multi-task learning; Weakly supervised learning
• ISSN: 1746-8094; 1746-8108
• DOI: 10.1016/j.bspc.2024.106398

As a hot topic in brain diseases, diagnostic and clinical score prediction of subjects based on multi-modal images helps assess pathological stages and estimate disease progression. Since brain atrophy occurs only in localized regions, previous patch-based deep learning methods require pre-determination of discriminative locations in the brain. In other words, the features extracted from the pre-determined potential atrophy locations are not fully adapted to the tasks in subsequent stages. Besides, most methods focus only on single-modal information with a single task, thus ignoring the intrinsic correlation between multi-modal information and multi-task variables. Furthermore, simply discarding subjects with incomplete clinical scores limits the number of available subjects. In this paper, we propose a multi-task joint learning network (MTJLN) for both brain disease diagnosis and clinical score prediction using multi-modal data and incomplete clinical scores. Specifically, we divided the brain images into 216 local patches covering all potential lesion locations. Then, an image patch pruning algorithm is designed for pruning the information-poor patches. The fine-grained multi-modal image features based on patches and coarse-grained non-image features are fused in the middle layer and used to predict multi-task variables. The ingenious design of the weighted loss function enables subjects with incomplete clinical scores to participate in network training. The experimental results of 842 subjects from the ADNI database demonstrate that the proposed method can effectively predict the pathological stage and clinical score of subjects. •Image patch pruning algorithm prunes information-poor local patches.•Reciprocity of categorical and scoring variables contributed to multitask prediction.•Multi-task weighted loss makes subjects with incomplete clinical scores available.•Hybrid fusion of fine-grained image features and coarse-grained non-image features.•Multi-modal features can model subjects more comprehensively.