Behavioural and computational evidence for memory consolidation biased by reward-prediction errors

• Published in: bioRxiv
• Main Authors: Roscow, Emma L, Jones, Matthew W, Lepora, Nathan F
• Format: Paper
• Language: English
• Published: Cold Spring Harbor Cold Spring Harbor Laboratory Press 26.07.2019
• Subjects: Computer applications; Hippocampus; Maze learning; Neocortex; Neostriatum; Reinforcement; Rodents; Sleep
• DOI: 10.1101/716290

Neural activity encoding recent experiences is replayed during sleep and rest to promote consolidation of the corresponding memories. However, precisely which features of experience influence replay prioritisation to optimise adaptive behaviour remains unclear. Here, we trained adult male rats on a novel maze-based reinforcement learning task designed to dissociate reward outcomes from reward-prediction errors. Four variations of a reinforcement learning model were fitted to the rats' behaviour over multiple days. Behaviour was best predicted by a model incorporating replay biased by reward-prediction error, compared to the same model with no replay; random replay or reward-biased replay produced poorer predictions of behaviour. This insight disentangles the influences of salience on replay, suggesting that reinforcement learning is tuned by post-learning replay biased by reward-prediction error, not by reward per se. This work therefore provides a behavioural and theoretical toolkit with which to measure and interpret replay in striatal, hippocampal and neocortical circuits. Footnotes * https://github.com/eroscow/QlearningReplay