The successor representation and temporal context

Samuel J. Gershman, Christopher D. Moore, Michael T. Todd, Kenneth Andrew Norman, Per B. Sederberg

Research output: Contribution to journalLetter

40 Scopus citations

Abstract

The successor representationwas introduced into reinforcement learning by Dayan (1993) as a means of facilitating generalization between states with similar successors. Although reinforcement learning in general has been used extensively as a model of psychological and neural processes, the psychological validity of the successor representation has yet to be explored. An interesting possibility is that the successor representation can be used not only for reinforcement learning but for episodic learning as well. Our main contribution is to show that a variant of the temporal context model (TCM; Howard & Kahana, 2002), an influential model of episodic memory, can be understood as directly estimating the successor representation using the temporal difference learning algorithm (Sutton & Barto, 1998). This insight leads to a generalization of TCM and new experimental predictions. In addition to casting a new normative light on TCM, this equivalence suggests a previously unexplored point of contact between different learning systems.

Original languageEnglish (US)
Pages (from-to)1553-1568
Number of pages16
JournalNeural computation
Volume24
Issue number6
DOIs
StatePublished - Dec 1 2012

All Science Journal Classification (ASJC) codes

  • Arts and Humanities (miscellaneous)
  • Cognitive Neuroscience

Fingerprint Dive into the research topics of 'The successor representation and temporal context'. Together they form a unique fingerprint.

  • Cite this

    Gershman, S. J., Moore, C. D., Todd, M. T., Norman, K. A., & Sederberg, P. B. (2012). The successor representation and temporal context. Neural computation, 24(6), 1553-1568. https://doi.org/10.1162/NECO_a_00282