A state representation for reinforcement learning and decision-making in the orbitofrontal cortex

Nicolas W. Schuck, Robert Wilson, Yael Niv

Research output: Chapter in Book/Report/Conference proceedingChapter

6 Scopus citations

Abstract

Despite decades of research, the exact ways in which the orbitofrontal cortex (OFC) influences cognitive function have remained mysterious. Anatomically, the OFC is characterized by remarkably broad connectivity to sensory, limbic, and subcortical areas, and functional studies have implicated the OFC in a plethora of functions ranging from facial processing to value-guided choice. Notwithstanding such diversity of findings, much research suggests that one important function of the OFC is to support decision-making and reinforcement learning. Here, we describe a novel theory that posits that OFC’s specific role in decision-making is to provide an up-to-date representation of task-related information, called a state representation. This representation reflects a mapping between distinct task states and sensory as well as unobservable information. We summarize evidence supporting the existence of such state representations in rodent and human OFC and argue that forming these state representations provides a crucial scaffold that allows animals to efficiently perform decision-making and reinforcement learning in high-dimensional and partially observable environments. Finally, we argue that our theory offers an integrating framework for linking the diversity of functions ascribed to OFC and is in line with its wide ranging connectivity.

Original languageEnglish (US)
Title of host publicationGoal-Directed Decision Making
Subtitle of host publicationComputations and Neural Circuits
PublisherElsevier
Pages259-278
Number of pages20
ISBN (Electronic)9780128120989
ISBN (Print)9780128120996
DOIs
StatePublished - Jan 1 2018

All Science Journal Classification (ASJC) codes

  • Neuroscience(all)

Keywords

  • Decision-making
  • Orbitofrontal cortex
  • Reinforcement learning

Fingerprint Dive into the research topics of 'A state representation for reinforcement learning and decision-making in the orbitofrontal cortex'. Together they form a unique fingerprint.

  • Cite this

    Schuck, N. W., Wilson, R., & Niv, Y. (2018). A state representation for reinforcement learning and decision-making in the orbitofrontal cortex. In Goal-Directed Decision Making: Computations and Neural Circuits (pp. 259-278). Elsevier. https://doi.org/10.1016/B978-0-12-812098-9.00012-7