Model-Based Influences on Humans' Choices and Striatal Prediction Errors

The mesostriatal dopamine system is prominently implicated in model-free reinforcement learning, with fMRI BOLD signals in ventral striatum notably covarying with model-free prediction errors. However, latent learning and devaluation studies show that behavior also shows hallmarks of model-based pla...

Full description

Saved in:
Bibliographic Details
Published inNeuron (Cambridge, Mass.) Vol. 69; no. 6; pp. 1204 - 1215
Main Authors Daw, Nathaniel D., Gershman, Samuel J., Seymour, Ben, Dayan, Peter, Dolan, Raymond J.
Format Journal Article
LanguageEnglish
Published United States Elsevier Inc 24.03.2011
Elsevier Limited
Subjects
Adult
Basal Ganglia - physiology
Behavior
Brain Mapping
Charitable foundations
Choice Behavior - physiology
Dopamine
Dopamine - metabolism
Female
Humans
Logistic Models
Magnetic Resonance Imaging
Male
Models, Neurological
Neurons - physiology
Neuropsychological Tests
Neurosciences
Reinforcement, Psychology
Studies
Valuation
Online AccessGet full text

Cover

More Information
Summary:The mesostriatal dopamine system is prominently implicated in model-free reinforcement learning, with fMRI BOLD signals in ventral striatum notably covarying with model-free prediction errors. However, latent learning and devaluation studies show that behavior also shows hallmarks of model-based planning, and the interaction between model-based and model-free values, prediction errors, and preferences is underexplored. We designed a multistep decision task in which model-based and model-free influences on human choice behavior could be distinguished. By showing that choices reflected both influences we could then test the purity of the ventral striatal BOLD signal as a model-free report. Contrary to expectations, the signal reflected both model-free and model-based predictions in proportions matching those that best explained choice behavior. These results challenge the notion of a separate model-free learner and suggest a more integrated computational architecture for high-level human decision-making. ► Humans learn both a world model and reinforcement-driven choice preferences ► BOLD responses in striatum and prefrontal cortex reflect both sorts of learning ► Across subjects, striatal BOLD tracks individual differences in model use ► Unexpectedly, ventral striatum shows combined model-based/free influences
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ObjectType-Article-2
ObjectType-Feature-1
ISSN:0896-6273
1097-4199
1097-4199
DOI:10.1016/j.neuron.2011.02.027