Deep sr-DDL: Deep structurally regularized dynamic dictionary learning to integrate multimodal and dynamic functional connectomics data for multidimensional clinical characterizations

• Published in: NeuroImage (Orlando, Fla.) Vol. 241; p. 118388
• Main Authors: D’Souza, N.S., Nebel, M.B., Crocetti, D., Robinson, J., Wymbs, N., Mostofsky, S.H., Venkataraman, A.
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 01.11.2021; Elsevier Limited; Elsevier
• Subjects: Alzheimer's disease; Autism; Autism Spectrum Disorder - diagnostic imaging; Autism Spectrum Disorder - physiopathology; Brain architecture; Brain mapping; Clinical severity; Cognition; Connectome - methods; Databases, Factual; Deep Learning; Dictionaries; Diffusion tensor imaging; Diffusion Tensor Imaging - methods; Dynamic dictionary learning; Functional magnetic resonance imaging; Humans; Image Processing, Computer-Assisted - methods; Magnetic Resonance Imaging - methods; Medical imaging; Multimodal Imaging - methods; Multimodal integration; Neural networks; Neural Networks, Computer; Neuroimaging; Schizophrenia; Structural regularization; Functional magnetic resonance imaging; Dynamic dictionary learning; Clinical severity; Multimodal integration; Structural regularization; Diffusion tensor imaging
• ISSN: 1053-8119; 1095-9572; 1095-9572
• DOI: 10.1016/j.neuroimage.2021.118388

•We integrate dynamic and multimodal connectomics data with behavior in a deep-generative hybrid framework.•We outperform various statistical, graph theoretic, and state-of-the-art deep learning baselines at behavioral prediction on two separate datasets.•We provide qualitative and quantitative comparisons of the learnt brain basis for both datasets.•We effectively fold in the anatomical information to track temporal changes during the resting state scan. [Display omitted] We propose a novel integrated framework that jointly models complementary information from resting-state functional MRI (rs-fMRI) connectivity and diffusion tensor imaging (DTI) tractography to extract biomarkers of brain connectivity predictive of behavior. Our framework couples a generative model of the connectomics data with a deep network that predicts behavioral scores. The generative component is a structurally-regularized Dynamic Dictionary Learning (sr-DDL) model that decomposes the dynamic rs-fMRI correlation matrices into a collection of shared basis networks and time varying subject-specific loadings. We use the DTI tractography to regularize this matrix factorization and learn anatomically informed functional connectivity profiles. The deep component of our framework is an LSTM-ANN block, which uses the temporal evolution of the subject-specific sr-DDL loadings to predict multidimensional clinical characterizations. Our joint optimization strategy collectively estimates the basis networks, the subject-specific time-varying loadings, and the neural network weights. We validate our framework on a dataset of neurotypical individuals from the Human Connectome Project (HCP) database to map to cognition and on a separate multi-score prediction task on individuals diagnosed with Autism Spectrum Disorder (ASD) in a five-fold cross validation setting. Our hybrid model outperforms several state-of-the-art approaches at clinical outcome prediction and learns interpretable multimodal neural signatures of brain organization.