Temporal Difference Models and Reward-Related Learning in the Human Brain

• Published in: Neuron (Cambridge, Mass.) Vol. 38; no. 2; pp. 329 - 337
• Main Authors: O'Doherty, John P., Dayan, Peter, Friston, Karl, Critchley, Hugo, Dolan, Raymond J.
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 24.04.2003; Elsevier Limited
• Subjects: Adolescent; Adult; Algorithms; Brain; Brain - anatomy & histology; Brain - physiology; Brain Mapping; Charitable foundations; Conditioning, Classical - physiology; Corpus Striatum - anatomy & histology; Corpus Striatum - physiology; Dopamine; Female; Frontal Lobe - anatomy & histology; Frontal Lobe - physiology; Humans; Learning - physiology; Magnetic Resonance Imaging; Male; Reference Values; Reflex, Pupillary - physiology; Reward; Taste - physiology; Time Perception - physiology
• ISSN: 0896-6273; 1097-4199
• DOI: 10.1016/S0896-6273(03)00169-7

Temporal difference learning has been proposed as a model for Pavlovian conditioning, in which an animal learns to predict delivery of reward following presentation of a conditioned stimulus (CS). A key component of this model is a prediction error signal, which, before learning, responds at the time of presentation of reward but, after learning, shifts its response to the time of onset of the CS. In order to test for regions manifesting this signal profile, subjects were scanned using event-related fMRI while undergoing appetitive conditioning with a pleasant taste reward. Regression analyses revealed that responses in ventral striatum and orbitofrontal cortex were significantly correlated with this error signal, suggesting that, during appetitive conditioning, computations described by temporal difference learning are expressed in the human brain.