Fusion of Multi-Modal Neuroimaging Data and Association With Cognitive Data

• Published in: 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) pp. 1 - 5
• Main Authors: LoPresto, Mark D., Akhonda, M. A. B. S., Calhoun, Vince D., Adali, Tulay
• Format: Conference Proceeding
• Language: English
• Published: IEEE 04.06.2023
• Subjects: Blind source separation; Cognition; Conferences; Data Fusion; Independent Vector Analysis; Mental disorders; MRI; Neuroimaging; Precision medicine; Psychological Assessment; Signal processing
• DOI: 10.1109/ICASSPW59220.2023.10193147

Efforts to develop a deeper understanding of the human brain benefit from the joint analysis (fusion) of multiple sources of data to exploit the complementary information in each modality. A number of Blind Source Separation (BSS) techniques have been developed for data-driven analysis and fusion of neuroimaging data, but only recently have fusion frameworks emerged that consider cognitive data alongside the neuroimaging data. In our approach, we first apply transposed Independent Vector Analysis (tIVA) across three neuroimaging modalities to extract subject covariations that are then associated with cognitive data. The tIVA step allows full interaction of the neuroimaging modalities to discover cross-modality relationships. We cluster the data based on the subject covariations to find the cognitive scores that offer significant discrimination between each pair of clusters. Our approach allows full interaction of multiple neuroimaging modalities and makes direct association with the cognitive data, identifying the Brief Assessment of Cognition in Schizophrenia Composite Score (BACS CS), Hopkins Verbal Learning Test (HVLT) and the Neuropsychological Assessment Battery (NAB) Mazes Test as being strongly associated with the cross-modality connections.