Visual and linguistic semantic representations are aligned at the border of human visual cortex

• Published in: Nature neuroscience Vol. 24; no. 11; pp. 1628 - 1636
• Main Authors: Popham, Sara F., Huth, Alexander G., Bilenko, Natalia Y., Deniz, Fatma, Gao, James S., Nunez-Elizalde, Anwar O., Gallant, Jack L.
• Format: Journal Article
• Language: English
• Published: New York Nature Publishing Group US 01.11.2021; Nature Publishing Group
• Subjects: 59/36; 631/378/116/2395; 631/378/2613; 631/378/2649/1594; 631/378/2649/1723; Adult; Animal Genetics and Genomics; Behavioral Sciences; Biological Techniques; Biomedical and Life Sciences; Biomedicine; Brain mapping; Brain research; Female; Functional magnetic resonance imaging; Humans; Linguistics - methods; Magnetic resonance imaging; Magnetic Resonance Imaging - methods; Male; Memory; Mental representation; Networks; Neurobiology; Neuroimaging; Neurosciences; Pattern Recognition, Visual - physiology; Photic Stimulation - methods; Psychological aspects; Selectivity; Semantic memory; Semantics; Visual cortex; Visual Cortex - diagnostic imaging; Visual Cortex - physiology; Visual perception; Young Adult; United States
• ISSN: 1097-6256; 1546-1726
• DOI: 10.1038/s41593-021-00921-6

Semantic information in the human brain is organized into multiple networks, but the fine-grain relationships between them are poorly understood. In this study, we compared semantic maps obtained from two functional magnetic resonance imaging experiments in the same participants: one that used silent movies as stimuli and another that used narrative stories. Movies evoked activity from a network of modality-specific, semantically selective areas in visual cortex. Stories evoked activity from another network of semantically selective areas immediately anterior to visual cortex. Remarkably, the pattern of semantic selectivity in these two distinct networks corresponded along the boundary of visual cortex: for visual categories represented posterior to the boundary, the same categories were represented linguistically on the anterior side. These results suggest that these two networks are smoothly joined to form one contiguous map. This study shows that visual areas pass information to the amodal semantic system through semantically selective channels aligned at the border of visual cortex. This architecture might support the integration of visual perception and semantic memory.