Operant Matching Is a Generic Outcome of Synaptic Plasticity Based on the Covariance between Reward and Neural Activity

• Published in: Proceedings of the National Academy of Sciences - PNAS Vol. 103; no. 41; pp. 15224 - 15229
• Main Authors: Loewenstein, Yonatan, Seung, H. Sebastian
• Format: Journal Article
• Language: English
• Published: United States National Academy of Sciences 10.10.2006; National Acad Sciences
• Subjects: Animals; Approximation; Behavior; Biological Sciences; Brain; Conditioning, Operant - physiology; Covariance; Decision making; Decision Making - physiology; Experimentation; Humans; Learning; Mathematical theorems; Modeling; Models, Neurological; Neuronal Plasticity - physiology; Neurons; Neurons - physiology; Neurosciences; Plasticity; Reward; Simulations; Stochastic models; Synapses; Synapses - physiology; Variance analysis
• ISSN: 0027-8424; 1091-6490
• DOI: 10.1073/pnas.0505220103

The probability of choosing an alternative in a long sequence of repeated choices is proportional to the total reward derived from that alternative, a phenomenon known as Herrnstein's matching law. This behavior is remarkably conserved across species and experimental conditions, but its underlying neural mechanisms still are unknown. Here, we propose a neural explanation of this empirical law of behavior. We hypothesize that there are forms of synaptic plasticity driven by the covariance between reward and neural activity and prove mathematically that matching is a generic outcome of such plasticity. Two hypothetical types of synaptic plasticity, embedded in decision-making neural network models, are shown to yield matching behavior in numerical simulations, in accord with our general theorem. We show how this class of models can be tested experimentally by making reward not only contingent on the choices of the subject but also directly contingent on fluctuations in neural activity. Maximization is shown to be a generic outcome of synaptic plasticity driven by the sum of the covariances between reward and all past neural activities.