Uncertainty in learning, choice, and visual fixation

Uncertainty plays a critical role in reinforcement learning and decision making. However, exactly how it influences behavior remains unclear. Multiarmed-bandit tasks offer an ideal test bed, since computational tools such as approximate Kalman filters can closely characterize the interplay between t...

Full description

Saved in:
Bibliographic Details
Published inProceedings of the National Academy of Sciences - PNAS Vol. 117; no. 6; pp. 3291 - 3300
Main Authors Stojić, Hrvoje, Orquin, Jacob L., Dayan, Peter, Dolan, Raymond J., Speekenbrink, Maarten
Format Journal Article
LanguageEnglish
Published United States National Academy of Sciences 11.02.2020
Subjects
Adolescent
Adult
Algorithms
Biological Sciences
Choice learning
Computer applications
Decision making
Decision Making - physiology
Exploratory Behavior - physiology
Female
Fixation
Fixation, Ocular - physiology
Humans
Kalman filters
Learning
Male
Reinforcement, Psychology
Social Sciences
Software
Task Performance and Analysis
Uncertainty
Visual discrimination learning
Young Adult
visual fixation
exploration–exploitation
uncertainty
decision making
reinforcement learning
Online AccessGet full text

Cover

More Information
Summary:Uncertainty plays a critical role in reinforcement learning and decision making. However, exactly how it influences behavior remains unclear. Multiarmed-bandit tasks offer an ideal test bed, since computational tools such as approximate Kalman filters can closely characterize the interplay between trial-by-trial values, uncertainty, learning, and choice. To gain additional insight into learning and choice processes, we obtained data from subjects’ overt allocation of gaze. The estimated value and estimation uncertainty of options influenced what subjects looked at before choosing; these same quantities also influenced choice, as additionally did fixation itself. A momentary measure of uncertainty in the form of absolute prediction errors determined how long participants looked at the obtained outcomes. These findings affirm the importance of uncertainty in multiple facets of behavior and help delineate its effects on decision making.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
2R.J.D. and M.S. contributed equally to this work.
Edited by Terrence J. Sejnowski, Salk Institute for Biological Studies, La Jolla, CA, and approved December 27, 2019 (received for review July 11, 2019)
Author contributions: H.S., J.L.O., P.D., and M.S. designed research; H.S. and J.L.O. performed research; H.S. analyzed data; H.S., J.L.O., P.D., and M.S. interpreted the results; H.S. prepared the paper; and H.S., J.L.O., P.D., R.J.D., and M.S. wrote the paper.
ISSN:0027-8424
1091-6490
1091-6490
DOI:10.1073/pnas.1911348117