Category-Selective Representation of Relationships in the Visual Cortex

• Published in: The Journal of neuroscience Vol. 44; no. 5; p. e0250232023
• Main Authors: Abassi, Etienne, Papeo, Liuba
• Format: Journal Article
• Language: English
• Published: United States Society for Neuroscience 31.01.2024
• Subjects: Brain Mapping - methods; Coding; Female; Functional magnetic resonance imaging; Human Body; Humans; Magnetic Resonance Imaging - methods; Male; Pattern analysis; Pattern classification; Pattern Recognition, Visual - physiology; Photic Stimulation - methods; Representations; Social behavior; Social factors; Visual cortex; Visual Cortex - physiology; Visual Perception - physiology; fMRI; social interaction; visual perception; social brain; relational information; social perception
• ISSN: 0270-6474; 1529-2401; 1529-2401
• DOI: 10.1523/JNEUROSCI.0250-23.2023

Understanding social interaction requires processing social agents and their relationships. The latest results show that much of this process is visually solved: visual areas can represent multiple people encoding emergent information about their interaction that is not explained by the response to the individuals alone. A neural signature of this process is an increased response in visual areas, to face-to-face (seemingly interacting) people, relative to people presented as unrelated (back-to-back). This effect highlighted a network of visual areas for representing relational information. Using functional MRI, we measured the brain activity of healthy female and male humans ( = 42), in response to images of two faces or two (head-blurred) bodies, facing toward or away from each other. Taking the effect as a signature of relation perception, we found that relations between faces and between bodies were coded in distinct areas, mirroring the categorical representation of faces and bodies in the visual cortex. Additional analyses suggest the existence of a third network encoding relations between (nonsocial) objects. Finally, a separate occipitotemporal network showed the generalization of relational information across body, face, and nonsocial object dyads (multivariate pattern classification analysis), revealing shared properties of relations across categories. In sum, beyond single entities, the visual cortex encodes the relations that bind multiple entities into relationships; it does so in a category-selective fashion, thus respecting a general organizing principle of representation in high-level vision. Visual areas encoding visual relational information can reveal the processing of emergent properties of social (and nonsocial) interaction, which trigger inferential processes.