Novelty and Inductive Generalization in Human Reinforcement Learning

Samuel J. Gershman, Yael Niv

Research output: Contribution to journalArticle

26 Scopus citations

Abstract

In reinforcement learning (RL), a decision maker searching for the most rewarding option is often faced with the question: What is the value of an option that has never been tried before? One way to frame this question is as an inductive problem: How can I generalize my previous experience with one set of options to a novel option? We show how hierarchical Bayesian inference can be used to solve this problem, and we describe an equivalence between the Bayesian model and temporal difference learning algorithms that have been proposed as models of RL in humans and animals. According to our view, the search for the best option is guided by abstract knowledge about the relationships between different options in an environment, resulting in greater search efficiency compared to traditional RL algorithms previously applied to human cognition. In two behavioral experiments, we test several predictions of our model, providing evidence that humans learn and exploit structured inductive knowledge to make predictions about novel options. In light of this model, we suggest a new interpretation of dopaminergic responses to novelty.

Original languageEnglish (US)
Pages (from-to)391-415
Number of pages25
JournalTopics in Cognitive Science
Volume7
Issue number3
DOIs
StatePublished - Jul 1 2015

All Science Journal Classification (ASJC) codes

  • Experimental and Cognitive Psychology
  • Linguistics and Language
  • Human-Computer Interaction
  • Cognitive Neuroscience
  • Artificial Intelligence

Keywords

  • Bayesian inference
  • Exploration-exploitation dilemma
  • Neophilia
  • Neophobia
  • Reinforcement learning

Fingerprint Dive into the research topics of 'Novelty and Inductive Generalization in Human Reinforcement Learning'. Together they form a unique fingerprint.

  • Cite this