Holistic Reinforcement Learning

The Role of Structure and Attention

Angela Radulescu, Yael Niv, Ian Ballard

Research output: Contribution to journalReview article

1 Citation (Scopus)

Abstract

Compact representations of the environment allow humans to behave efficiently in a complex world. Reinforcement learning models capture many behavioral and neural effects but do not explain recent findings showing that structure in the environment influences learning. In parallel, Bayesian cognitive models predict how humans learn structured knowledge but do not have a clear neurobiological implementation. We propose an integration of these two model classes in which structured knowledge learned via approximate Bayesian inference acts as a source of selective attention. In turn, selective attention biases reinforcement learning towards relevant dimensions of the environment. An understanding of structure learning will help to resolve the fundamental challenge in decision science: explaining why people make the decisions they do.

Original languageEnglish (US)
Pages (from-to)278-292
Number of pages15
JournalTrends in Cognitive Sciences
Volume23
Issue number4
DOIs
StatePublished - Apr 1 2019

Fingerprint

Learning
Reinforcement (Psychology)

All Science Journal Classification (ASJC) codes

  • Neuropsychology and Physiological Psychology
  • Experimental and Cognitive Psychology
  • Cognitive Neuroscience

Cite this

Radulescu, Angela ; Niv, Yael ; Ballard, Ian. / Holistic Reinforcement Learning : The Role of Structure and Attention. In: Trends in Cognitive Sciences. 2019 ; Vol. 23, No. 4. pp. 278-292.
@article{20428732d29e437e8f1ed0672e6577a1,
title = "Holistic Reinforcement Learning: The Role of Structure and Attention",
abstract = "Compact representations of the environment allow humans to behave efficiently in a complex world. Reinforcement learning models capture many behavioral and neural effects but do not explain recent findings showing that structure in the environment influences learning. In parallel, Bayesian cognitive models predict how humans learn structured knowledge but do not have a clear neurobiological implementation. We propose an integration of these two model classes in which structured knowledge learned via approximate Bayesian inference acts as a source of selective attention. In turn, selective attention biases reinforcement learning towards relevant dimensions of the environment. An understanding of structure learning will help to resolve the fundamental challenge in decision science: explaining why people make the decisions they do.",
author = "Angela Radulescu and Yael Niv and Ian Ballard",
year = "2019",
month = "4",
day = "1",
doi = "10.1016/j.tics.2019.01.010",
language = "English (US)",
volume = "23",
pages = "278--292",
journal = "Trends in Cognitive Sciences",
issn = "1364-6613",
publisher = "Elsevier Limited",
number = "4",

}

Holistic Reinforcement Learning : The Role of Structure and Attention. / Radulescu, Angela; Niv, Yael; Ballard, Ian.

In: Trends in Cognitive Sciences, Vol. 23, No. 4, 01.04.2019, p. 278-292.

Research output: Contribution to journalReview article

TY - JOUR

T1 - Holistic Reinforcement Learning

T2 - The Role of Structure and Attention

AU - Radulescu, Angela

AU - Niv, Yael

AU - Ballard, Ian

PY - 2019/4/1

Y1 - 2019/4/1

N2 - Compact representations of the environment allow humans to behave efficiently in a complex world. Reinforcement learning models capture many behavioral and neural effects but do not explain recent findings showing that structure in the environment influences learning. In parallel, Bayesian cognitive models predict how humans learn structured knowledge but do not have a clear neurobiological implementation. We propose an integration of these two model classes in which structured knowledge learned via approximate Bayesian inference acts as a source of selective attention. In turn, selective attention biases reinforcement learning towards relevant dimensions of the environment. An understanding of structure learning will help to resolve the fundamental challenge in decision science: explaining why people make the decisions they do.

AB - Compact representations of the environment allow humans to behave efficiently in a complex world. Reinforcement learning models capture many behavioral and neural effects but do not explain recent findings showing that structure in the environment influences learning. In parallel, Bayesian cognitive models predict how humans learn structured knowledge but do not have a clear neurobiological implementation. We propose an integration of these two model classes in which structured knowledge learned via approximate Bayesian inference acts as a source of selective attention. In turn, selective attention biases reinforcement learning towards relevant dimensions of the environment. An understanding of structure learning will help to resolve the fundamental challenge in decision science: explaining why people make the decisions they do.

UR - http://www.scopus.com/inward/record.url?scp=85062018846&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85062018846&partnerID=8YFLogxK

U2 - 10.1016/j.tics.2019.01.010

DO - 10.1016/j.tics.2019.01.010

M3 - Review article

VL - 23

SP - 278

EP - 292

JO - Trends in Cognitive Sciences

JF - Trends in Cognitive Sciences

SN - 1364-6613

IS - 4

ER -