Inferring relevance in a changing world

Robert C. Wilson, Yael Niv

Research output: Contribution to journalArticle

50 Scopus citations

Abstract

Reinforcement learning models of human and animal learning usually concentrate on how we learn the relationship between different stimuli or actions and rewards. However, in real-world situations "stimuli" are ill-defined. On the one hand, our immediate environment is extremely multidimensional. On the other hand, in every decision making scenario only a few aspects of the environment are relevant for obtaining reward, while most are irrelevant. Thus a key question is how do we learn these relevant dimensions, that is, how do we learn what to learn about? We investigated this process of "representation learning" experimentally, using a task in which one stimulus dimension was relevant for determining reward at each point in time. As in real life situations, in our task the relevant dimension can change without warning, adding ever-present uncertainty engendered by a constantly changing environment. We show that human performance on this task is better described by a suboptimal strategy based on selective attention and serial-hypothesis-testing rather than a normative strategy based on probabilistic inference. From this, we conjecture that the problem of inferring relevance in general scenarios is too computationally demanding for the brain to solve optimally. As a result the brain utilizes approximations, employing these even in simplified scenarios in which optimal representation learning is tractable, such as the one in our experiment.

Original languageEnglish (US)
Pages (from-to)1-14
Number of pages14
JournalFrontiers in Human Neuroscience
Issue numberJANUARY 2012
DOIs
StatePublished - Jan 24 2012

All Science Journal Classification (ASJC) codes

  • Neuropsychology and Physiological Psychology
  • Neurology
  • Psychiatry and Mental health
  • Biological Psychiatry
  • Behavioral Neuroscience

Keywords

  • Bayesian inference
  • Decision making
  • Reinforcement learning
  • Representation learning
  • Selective attention

Fingerprint Dive into the research topics of 'Inferring relevance in a changing world'. Together they form a unique fingerprint.

  • Cite this