Neural population control via deep image synthesis

• Published in: Science (American Association for the Advancement of Science) Vol. 364; no. 6439; p. 453
• Main Authors: Bashivan, Pouya, Kar, Kohitij, DiCarlo, James J.
• Format: Journal Article
• Language: English
• Published: United States American Association for the Advancement of Science 03.05.2019; The American Association for the Advancement of Science
• Subjects: Activation; Animal behavior; Animal models; Animals; Artificial neural networks; Black boxes; Brain; Controllers; Eye (anatomy); Firing pattern; Information processing; Knowledge; Learning theory; Light effects; Macaca; Machine learning; Mathematical models; Model matching; Model testing; Models, Neurological; Nerve Net - physiology; Nervous system; Networks; Neural networks; Neural Networks, Computer; Neuroimaging; Neurons; Neurons - physiology; Neurophysiology; Object recognition; Opacity; Population control; Predictive control; RESEARCH ARTICLE SUMMARY; Stability; Stimuli; Synthesis; Therapeutic applications; Visual cortex; Visual Cortex - physiology; Visual effects; Visual Fields - physiology; Visual stimuli; Visual system
• ISSN: 0036-8075; 1095-9203; 1095-9203
• DOI: 10.1126/science.aav9436

To what extent are predictive deep learning models of neural responses useful for generating experimental hypotheses? Bashivan et al. took an artificial neural network built to model the behavior of the target visual system and used it to construct images predicted to either broadly activate large populations of neurons or selectively activate one population while keeping the others unchanged. They then analyzed the effectiveness of these images in producing the desired effects in the macaque visual cortex. The manipulations showed very strong effects and achieved considerable and highly selective influence over the neuronal populations. Using novel and non-naturalistic images, the neural network was shown to reproduce the overall behavior of the animals' neural responses. Science , this issue p. eaav9436 A deep artificial neural network can model primate vision. Particular deep artificial neural networks (ANNs) are today’s most accurate models of the primate brain’s ventral visual stream. Using an ANN-driven image synthesis method, we found that luminous power patterns (i.e., images) can be applied to primate retinae to predictably push the spiking activity of targeted V4 neural sites beyond naturally occurring levels. This method, although not yet perfect, achieves unprecedented independent control of the activity state of entire populations of V4 neural sites, even those with overlapping receptive fields. These results show how the knowledge embedded in today’s ANN models might be used to noninvasively set desired internal brain states at neuron-level resolution, and suggest that more accurate ANN models would produce even more accurate control.