The algorithmic anatomy of model-based evaluation

Nathaniel D. Daw, Peter Dayan

Research output: Contribution to journalArticlepeer-review

77 Scopus citations

Abstract

Despite many debates in the first half of the twentieth century, it is now largely a truism that humans and other animals build models of their environments and use them for prediction and control. However, model-based (MB) reasoning presents severe computational challenges. Alternative, computationally simpler, model-free (MF) schemes have been suggested in the reinforcement learning literature, and have afforded influential accounts of behavioural and neural data. Here, we study the realization of MB calculations, and the ways that this might be woven together with MF values and evaluation methods. There are as yet mostly only hints in the literature as to the resulting tapestry, so we offer more preview than review.

Original languageEnglish (US)
JournalPhilosophical Transactions of the Royal Society B: Biological Sciences
Volume369
Issue number1655
DOIs
StatePublished - Nov 5 2014

All Science Journal Classification (ASJC) codes

  • Biochemistry, Genetics and Molecular Biology(all)
  • Agricultural and Biological Sciences(all)

Keywords

  • Model-based reasoning
  • Model-free reasoning
  • Monte Carlo tree search
  • Orbitofrontal cortex
  • Reinforcement learning
  • Striatum

Fingerprint Dive into the research topics of 'The algorithmic anatomy of model-based evaluation'. Together they form a unique fingerprint.

Cite this