Reconstructing Retinal Visual Images from 3T fMRI Data Enhanced by Unsupervised Learning

• Published in: Proceedings (International Symposium on Biomedical Imaging) Vol. 2024; pp. 1 - 5
• Main Authors: Xiong, Yujian, Zhu, Wenhui, Lu, Zhong-Lin, Wang, Yalin
• Format: Conference Proceeding Journal Article
• Language: English
• Published: United States IEEE 01.05.2024
• Subjects: Data models; Functional magnetic resonance imaging; Generative Adversarial Network; Generative adversarial networks; Task analysis; Training; Visual Cortex; Visual systems; Visualization; Visual Cortex; Functional Magnetic Resonance Imaging; Generative Adversarial Network
• ISSN: 1945-7928; 1945-8452
• DOI: 10.1109/ISBI56570.2024.10635641

The reconstruction of human visual inputs from brain activity, particularly through functional Magnetic Resonance Imaging (fMRI), holds promising avenues for unraveling the mechanisms of the human visual system. Despite the significant strides made by deep learning methods in improving the quality and interpretability of visual reconstruction, there remains a substantial demand for high-quality, long-duration, subject-specific 7-Tesla fMRI experiments. The challenge arises in integrating diverse smaller 3-Tesla datasets or accommodating new subjects with brief and low-quality fMRI scans. In response to these constraints, we propose a novel framework that generates enhanced 3T fMRI data through an unsupervised Generative Adversarial Network (GAN), leveraging unpaired training across two distinct fMRI datasets in 7T and 3T, respectively. This approach aims to overcome the limitations of the scarcity of high-quality 7-Tesla data and the challenges associated with brief and low-quality scans in 3-Tesla experiments. In this paper, we demonstrate the reconstruction capabilities of the enhanced 3T fMRI data, highlighting its proficiency in generating superior input visual images compared to data-intensive methods trained and tested on a single subject.