Perceived Image Decoding From Brain Activity Using Shared Information of Multi-Subject fMRI Data

• Published in: IEEE access Vol. 9; pp. 26593 - 26606
• Main Authors: Akamatsu, Yusuke, Harakawa, Ryosuke, Ogawa, Takahiro, Haseyama, Miki
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Bayes methods; Bayesian inference; Brain; Brain decoding; Brain modeling; Categories; Data acquisition; Data models; Decoding; Functional magnetic resonance imaging; functional magnetic resonance imaging (fMRI); latent variable model; Magnetic resonance imaging; multi-subject fMRI data; Representations; Semantics; Visualization
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2021.3057800

Decoding a person's cognitive contents from evoked brain activity is becoming important in the field of brain-computer interaction. Previous studies have decoded a perceived image from functional magnetic resonance imaging (fMRI) activity by constructing brain decoding models that were trained with a single subject's fMRI data. However, accurate decoding is still challenging since fMRI data acquired from only a single subject have several disadvantageous characteristics such as small sample size, noisy nature, and high dimensionality. In this article, we propose a method to decode categories of perceived images from fMRI activity using shared information of multi-subject fMRI data. Specifically, by aggregating fMRI data of multiple subjects that contain a large number of samples, we extract a low-dimensional latent representation shared by multi-subject fMRI data. Then the latent representation is nonlinearly transformed into visual features and semantic features of the perceived images to identify categories from various candidate categories. Our approach leverages rich information obtained from multi-subject fMRI data and improves the decoding performance. Experimental results obtained by using two public fMRI datasets showed that the proposed method can more accurately decode categories of perceived images from fMRI activity than previous approaches using a single subject's fMRI data.