Long-term cognitive decline prediction based on multi-modal data using Multimodal3DSiameseNet: transfer learning from Alzheimer’s disease to Parkinson’s disease

• Published in: International journal for computer assisted radiology and surgery Vol. 18; no. 5; pp. 809 - 818
• Main Authors: Ostertag, Cécilia, Visani, Muriel, Urruty, Thierry, Beurton-Aimar, Marie
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 01.05.2023; Springer Nature B.V; Springer Verlag
• Subjects: Alzheimer Disease - diagnostic imaging; Alzheimer's disease; Artificial neural networks; Brain; Cognitive Dysfunction - diagnosis; Cognitive Dysfunction - etiology; Computer Imaging; Computer Science; Datasets; Disease Progression; Health Informatics; Humans; Image filters; Imaging; Knowledge management; Machine Learning; Magnetic resonance imaging; Magnetic Resonance Imaging - methods; Medicine; Medicine & Public Health; Modal data; Neurodegenerative Diseases; Original; Original Article; Parkinson Disease - diagnostic imaging; Parkinson's disease; Pattern Recognition and Graphics; Radiology; Surgery; Time lag; Vision; Cognitive decline detection; Alzheimer’s disease; Transfer learning; Parkinson’s disease; Multimodal learning; Siamese deep neural network
• ISSN: 1861-6429; 1861-6410; 1861-6429
• DOI: 10.1007/s11548-023-02866-6

Purpose Monitoring and predicting the cognitive state of subjects with neurodegenerative disorders is crucial to provide appropriate treatment as soon as possible. In this work, we present a machine learning approach using multimodal data (brain MRI and clinical) from two early medical visits, to predict the longer-term cognitive decline of patients. Using transfer learning, our model can be successfully transferred from one neurodegenerative disease (Alzheimer’s) to another (Parkinson’s). Methods Our model is a Deep Neural Network with siamese sub-modules dedicated to extracting features from each modality. We pre-train it with data from ADNI (Alzheimer’s disease), then transfer it on the smaller PPMI dataset (Parkinson’s disease). We show that, even when we do not fine-tune the filters learnt from the ADNI MRIs, the transferred model’s results are satisfying on PPMI. Results The first main result is that our model provides satisfying long-term predictions of cognitive decline from any pair of early visits, with no fixed time delay between these visits (provided the potential decline has started at the second visit). The second main result is that the prediction performance on Parkinson’s dataset (PPMI) reaches an AUC of 0.81 on PPMI after transfer learning from Alzheimer’s dataset (ADNI), without even having to re-train the image filters, versus an AUC of 0.72 for the model trained from scratch on PPMI. Conclusions First, our model is effective for predicting long-term cognitive decline from only two visits, even with irregular intervals of time. When dealing with neurodegenerative diseases, where patients often miss some control visits, this is an important finding. Second, our model is able to transfer the knowledge learnt from one neurodegenerative disease (Alzheimer’s) to another (Parkinson’s), when using the same imaging modalities (brain MRI) and different clinical variables. This makes it usable even for diseases that are rare or under-studied.