Recurrent computations for visual pattern completion

• Published in: Proceedings of the National Academy of Sciences - PNAS Vol. 115; no. 35; pp. 8835 - 8840
• Main Authors: Tang, Hanlin, Schrimpf, Martin, Lotter, William, Moerman, Charlotte, Paredes, Ana, Caro, Josue Ortega, Hardesty, Walter, Cox, David, Kreiman, Gabriel
• Format: Journal Article
• Language: English
• Published: United States National Academy of Sciences 28.08.2018
• Subjects: Adolescent; Adult; Artificial intelligence; Biological Sciences; Cognition; Computation; Computer applications; Computer Simulation; Female; Human behavior; Humans; Male; Masking; Mathematical models; Models, Neurological; Neurosciences; Object recognition; Pattern recognition; Pattern recognition systems; Pattern Recognition, Visual - physiology; Physiological effects; Physiological responses; Physiology; Psychophysics; Social Sciences; Visibility; Visual aspects; Visual cortex; Visual perception; Visualization; computational neuroscience; pattern completion; machine learning; visual object recognition; artificial intelligence
• ISSN: 0027-8424; 1091-6490; 1091-6490
• DOI: 10.1073/pnas.1719397115

Making inferences from partial information constitutes a critical aspect of cognition. During visual perception, pattern completion enables recognition of poorly visible or occluded objects. We combined psychophysics, physiology, and computational models to test the hypothesis that pattern completion is implemented by recurrent computations and present three pieces of evidence that are consistent with this hypothesis. First, subjects robustly recognized objects even when they were rendered <15% visible, but recognition was largely impaired when processing was interrupted by backward masking. Second, invasive physiological responses along the human ventral cortex exhibited visually selective responses to partially visible objects that were delayed compared with whole objects, suggesting the need for additional computations. These physiological delays were correlated with the effects of backward masking. Third, state-of-the-art feed-forward computational architectures were not robust to partial visibility. However, recognition performance was recovered when the model was augmented with attractor-based recurrent connectivity. The recurrent model was able to predict which images of heavily occluded objects were easier or harder for humans to recognize, could capture the effect of introducing a backward mask on recognition behavior, and was consistent with the physiological delays along the human ventral visual stream. These results provide a strong argument of plausibility for the role of recurrent computations in making visual inferences from partial information.