The Curse of Planning: Dissecting Multiple Reinforcement-Learning Systems by Taxing the Central Executive

A. Ross Otto, Samuel J. Gershman, Arthur B. Markman, Nathaniel D. Daw

Research output: Contribution to journalArticlepeer-review

234 Scopus citations

Abstract

A number of accounts of human and animal behavior posit the operation of parallel and competing valuation systems in the control of choice behavior. In these accounts, a flexible but computationally expensive model-based reinforcement-learning system has been contrasted with a less flexible but more efficient model-free reinforcement-learning system. The factors governing which system controls behavior-and under what circumstances-are still unclear. Following the hypothesis that model-based reinforcement learning requires cognitive resources, we demonstrated that having human decision makers perform a demanding secondary task engenders increased reliance on a model-free reinforcement-learning strategy. Further, we showed that, across trials, people negotiate the trade-off between the two systems dynamically as a function of concurrent executive-function demands, and people's choice latencies reflect the computational expenses of the strategy they employ. These results demonstrate that competition between multiple learning systems can be controlled on a trial-by-trial basis by modulating the availability of cognitive resources.

Original languageEnglish (US)
Pages (from-to)751-761
Number of pages11
JournalPsychological Science
Volume24
Issue number5
DOIs
StatePublished - May 2013
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • General Psychology

Keywords

  • cognitive neuroscience
  • decision making

Fingerprint

Dive into the research topics of 'The Curse of Planning: Dissecting Multiple Reinforcement-Learning Systems by Taxing the Central Executive'. Together they form a unique fingerprint.

Cite this