Linear reinforcement learning in planning, grid fields, and cognitive control

Payam Piray, Nathaniel D. Daw

Research output: Contribution to journalArticlepeer-review

32 Scopus citations

Abstract

It is thought that the brain’s judicious reuse of previous computation underlies our ability to plan flexibly, but also that inappropriate reuse gives rise to inflexibilities like habits and compulsion. Yet we lack a complete, realistic account of either. Building on control engineering, here we introduce a model for decision making in the brain that reuses a temporally abstracted map of future events to enable biologically-realistic, flexible choice at the expense of specific, quantifiable biases. It replaces the classic nonlinear, model-based optimization with a linear approximation that softly maximizes around (and is weakly biased toward) a default policy. This solution demonstrates connections between seemingly disparate phenomena across behavioral neuroscience, notably flexible replanning with biases and cognitive control. It also provides insight into how the brain can represent maps of long-distance contingencies stably and componentially, as in entorhinal response fields, and exploit them to guide choice even under changing goals.

Original languageEnglish (US)
Article number4942
JournalNature communications
Volume12
Issue number1
DOIs
StatePublished - Dec 1 2021

All Science Journal Classification (ASJC) codes

  • General Chemistry
  • General Biochemistry, Genetics and Molecular Biology
  • General Physics and Astronomy

Fingerprint

Dive into the research topics of 'Linear reinforcement learning in planning, grid fields, and cognitive control'. Together they form a unique fingerprint.

Cite this