Multimodal Data Fusion Using Source Separation: Application to Medical Imaging

• Published in: Proceedings of the IEEE Vol. 103; no. 9; pp. 1494 - 1506
• Main Authors: Adali, Tulay, Levin-Schwartz, Yuri, Calhoun, Vince D.
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.09.2015; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Biomedical imaging; Brain models; Data integration; Data models; Electroencephalography; Magnetic resonance imaging; Multimodal sensors; NMR; Nuclear magnetic resonance; electroencephalography (EEG); independent component analysis (ICA); MRI; independent vector analysis (IVA); functional magnetic resonance imaging (fMRI); Data fusion; multimodality; medical imaging; source separation
• ISSN: 0018-9219; 1558-2256
• DOI: 10.1109/JPROC.2015.2461601

The joint independent component analysis (jICA) and the transposed independent vector analysis (tIVA) models are two effective solutions based on blind source separation (BSS) that enable fusion of data from multiple modalities in a symmetric and fully multivariate manner. The previous paper in this special issue discusses the properties and the main issues in the implementation of these two models. In this accompanying paper, we consider the application of these two models to fusion of multimodal medical imaging data-functional magnetic resonance imaging (fMRI), structural MRI (sMRI), and electroencephalography (EEG) data collected from a group of healthy controls and patients with schizophrenia performing an auditory oddball task. We show how both models can be used to identify a set of components that report on differences between the two groups, jointly, for all the modalities used in the study. We discuss the importance of algorithm and order selection as well as tradeoffs involved in the selection of one model over another. We note that for the selected data set, especially given the limited number of subjects available for the study, jICA provides a more desirable solution, however the use of an ICA algorithm that uses flexible density matching provides advantages over the most widely used algorithm, Infomax, for the problem.