A unifying framework for functional organization in early and higher ventral visual cortex

• Published in: Neuron (Cambridge, Mass.) Vol. 112; no. 14; pp. 2435 - 2451.e7
• Main Authors: Margalit, Eshed, Lee, Hyodong, Finzi, Dawn, DiCarlo, James J., Grill-Spector, Kalanit, Yamins, Daniel L.K.
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 17.07.2024
• Subjects: dimensionality; neural network; topography; ventral visual cortex; vision; wiring length; vision; ventral visual cortex; dimensionality; neural network; wiring length; topography
• ISSN: 0896-6273; 1097-4199; 1097-4199
• DOI: 10.1016/j.neuron.2024.04.018

A key feature of cortical systems is functional organization: the arrangement of functionally distinct neurons in characteristic spatial patterns. However, the principles underlying the emergence of functional organization in the cortex are poorly understood. Here, we develop the topographic deep artificial neural network (TDANN), the first model to predict several aspects of the functional organization of multiple cortical areas in the primate visual system. We analyze the factors driving the TDANN’s success and find that it balances two objectives: learning a task-general sensory representation and maximizing the spatial smoothness of responses according to a metric that scales with cortical surface area. In turn, the representations learned by the TDANN are more brain-like than in spatially unconstrained models. Finally, we provide evidence that the TDANN’s functional organization balances performance with between-area connection length. Our results offer a unified principle for understanding the functional organization of the primate ventral visual system. •Single model predicts function and spatial structure in early and higher visual cortex•Best model uses self-supervised learning and a scalable spatial constraint•More brain-like responses in spatially accurate than spatially unconstrained models•The local spatial constraint results in lower between-area wiring length Margalit et al. develop a topographic artificial neural network that predicts both functional responses and spatial organization of multiple cortical areas of the primate visual system. In turn, the model minimizes between-area wiring length and produces more brain-like responses to visual stimuli than spatially unconstrained alternative models.