Reinforcement Learning and Episodic Memory in Humans and Animals: An Integrative Framework

Samuel J. Gershman, Nathaniel D. Daw

Research output: Contribution to journalArticlepeer-review

267 Scopus citations


We review the psychology and neuroscience of reinforcement learning (RL), which has experienced significant progress in the past two decades, enabled by the comprehensive experimental study of simple learning and decision-making tasks. However, one challenge in the study of RL is computational: The simplicity of these tasks ignores important aspects of reinforcement learning in the real world: (a) State spaces are high-dimensional, continuous, and partially observable; this implies that (b) data are relatively sparse and, indeed, precisely the same situation may never be encountered twice; furthermore, (c) rewards depend on the long-term consequences of actions in ways that violate the classical assumptions that make RL tractable. A seemingly distinct challenge is that, cognitively, theories of RL have largely involved procedural and semantic memory, the way in which knowledge about action values or world models extracted gradually from many experiences can drive choice. This focus on semantic memory leaves out many aspects of memory, such as episodic memory, related to the traces of individual events. We suggest that these two challenges are related. The computational challenge can be dealt with, in part, by endowing RL systems with episodic memory, allowing them to (a) efficiently approximate value functions over complex state spaces, (b) learn with very little data, and (c) bridge long-term dependencies between actions and rewards. We review the computational theory underlying this proposal and the empirical evidence to support it. Our proposal suggests that the ubiquitous and diverse roles of memory in RL may function as part of an integrated learning system.

Original languageEnglish (US)
Pages (from-to)101-128
Number of pages28
JournalAnnual review of psychology
StatePublished - Jan 3 2017

All Science Journal Classification (ASJC) codes

  • General Psychology


  • Decision making
  • Memory
  • Reinforcement learning


Dive into the research topics of 'Reinforcement Learning and Episodic Memory in Humans and Animals: An Integrative Framework'. Together they form a unique fingerprint.

Cite this