A sensory–motor theory of the neocortex

• Published in: Nature neuroscience Vol. 27; no. 7; pp. 1221 - 1235
• Main Author: Rao, Rajesh P. N.
• Format: Journal Article
• Language: English
• Published: New York Nature Publishing Group 01.07.2024
• Subjects: Cerebral cortex; Coding; Computational neuroscience; Cortex (motor); Eye movements; Family trees; Hierarchies; Neocortex; Neural coding; Somatosensory cortex
• ISSN: 1097-6256; 1546-1726; 1546-1726
• DOI: 10.1038/s41593-024-01673-9

Recent neurophysiological and neuroanatomical studies suggest a close interaction between sensory and motor processes across the neocortex. Here, I propose that the neocortex implements active predictive coding (APC): each cortical area estimates both latent sensory states and actions (including potentially abstract actions internal to the cortex), and the cortex as a whole predicts the consequences of actions at multiple hierarchical levels. Feedback from higher areas modulates the dynamics of state and action networks in lower areas. I show how the same APC architecture can explain (1) how we recognize an object and its parts using eye movements, (2) why perception seems stable despite eye movements, (3) how we learn compositional representations, for example, part–whole hierarchies, (4) how complex actions can be planned using simpler actions, and (5) how we form episodic memories of sensory–motor experiences and learn abstract concepts such as a family tree. I postulate a mapping of the APC model to the laminar architecture of the cortex and suggest possible roles for cortico–cortical and cortico–subcortical pathways.Recent studies suggest a close interaction between sensory and motor processes across the neocortex. In this Perspective, Rao proposes active predictive coding as a sensory–motor theory that explains the structure of the neocortex as well as some of its diverse computational capabilities.