Neural Signatures of Prediction Errors in a Decision-Making Task Are Modulated by Action Execution Failures

Samuel D. McDougle, Peter A. Butcher, Darius E. Parvin, Fasial Mushtaq, Yael Niv, Richard B. Ivry, Jordan A. Taylor

Research output: Contribution to journalArticle

Abstract

Decisions must be implemented through actions, and actions are prone to error. As such, when an expected outcome is not obtained, an individual should be sensitive to not only whether the choice itself was suboptimal but also whether the action required to indicate that choice was executed successfully. The intelligent assignment of credit to action execution versus action selection has clear ecological utility for the learner. To explore this, we used a modified version of a classic reinforcement learning task in which feedback indicated whether negative prediction errors were, or were not, associated with execution errors. Using fMRI, we asked if prediction error computations in the human striatum, a key substrate in reinforcement learning and decision making, are modulated when a failure in action execution results in the negative outcome. Participants were more tolerant of non-rewarded outcomes when these resulted from execution errors versus when execution was successful, but reward was withheld. Consistent with this behavior, a model-driven analysis of neural activity revealed an attenuation of the signal associated with negative reward prediction errors in the striatum following execution failures. These results converge with other lines of evidence suggesting that prediction errors in the mesostriatal dopamine system integrate high-level information during the evaluation of instantaneous reward outcomes.

Original languageEnglish (US)
Pages (from-to)1606-1613.e5
JournalCurrent Biology
Volume29
Issue number10
DOIs
StatePublished - May 20 2019

Fingerprint

Reward
decision making
Decision Making
Decision making
prediction
Learning
learning
Reinforcement learning
Dopamine
credit
dopamine
Magnetic Resonance Imaging
Reinforcement (Psychology)
Feedback
Substrates

All Science Journal Classification (ASJC) codes

  • Biochemistry, Genetics and Molecular Biology(all)
  • Agricultural and Biological Sciences(all)

Keywords

  • decision making
  • error
  • reaching
  • reinforcement learning
  • striatum

Cite this

McDougle, Samuel D. ; Butcher, Peter A. ; Parvin, Darius E. ; Mushtaq, Fasial ; Niv, Yael ; Ivry, Richard B. ; Taylor, Jordan A. / Neural Signatures of Prediction Errors in a Decision-Making Task Are Modulated by Action Execution Failures. In: Current Biology. 2019 ; Vol. 29, No. 10. pp. 1606-1613.e5.
@article{9ae76264fa0a4d9ebdb9757775f42c8d,
title = "Neural Signatures of Prediction Errors in a Decision-Making Task Are Modulated by Action Execution Failures",
abstract = "Decisions must be implemented through actions, and actions are prone to error. As such, when an expected outcome is not obtained, an individual should be sensitive to not only whether the choice itself was suboptimal but also whether the action required to indicate that choice was executed successfully. The intelligent assignment of credit to action execution versus action selection has clear ecological utility for the learner. To explore this, we used a modified version of a classic reinforcement learning task in which feedback indicated whether negative prediction errors were, or were not, associated with execution errors. Using fMRI, we asked if prediction error computations in the human striatum, a key substrate in reinforcement learning and decision making, are modulated when a failure in action execution results in the negative outcome. Participants were more tolerant of non-rewarded outcomes when these resulted from execution errors versus when execution was successful, but reward was withheld. Consistent with this behavior, a model-driven analysis of neural activity revealed an attenuation of the signal associated with negative reward prediction errors in the striatum following execution failures. These results converge with other lines of evidence suggesting that prediction errors in the mesostriatal dopamine system integrate high-level information during the evaluation of instantaneous reward outcomes.",
keywords = "decision making, error, reaching, reinforcement learning, striatum",
author = "McDougle, {Samuel D.} and Butcher, {Peter A.} and Parvin, {Darius E.} and Fasial Mushtaq and Yael Niv and Ivry, {Richard B.} and Taylor, {Jordan A.}",
year = "2019",
month = "5",
day = "20",
doi = "10.1016/j.cub.2019.04.011",
language = "English (US)",
volume = "29",
pages = "1606--1613.e5",
journal = "Current Biology",
issn = "0960-9822",
publisher = "Cell Press",
number = "10",

}

Neural Signatures of Prediction Errors in a Decision-Making Task Are Modulated by Action Execution Failures. / McDougle, Samuel D.; Butcher, Peter A.; Parvin, Darius E.; Mushtaq, Fasial; Niv, Yael; Ivry, Richard B.; Taylor, Jordan A.

In: Current Biology, Vol. 29, No. 10, 20.05.2019, p. 1606-1613.e5.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Neural Signatures of Prediction Errors in a Decision-Making Task Are Modulated by Action Execution Failures

AU - McDougle, Samuel D.

AU - Butcher, Peter A.

AU - Parvin, Darius E.

AU - Mushtaq, Fasial

AU - Niv, Yael

AU - Ivry, Richard B.

AU - Taylor, Jordan A.

PY - 2019/5/20

Y1 - 2019/5/20

N2 - Decisions must be implemented through actions, and actions are prone to error. As such, when an expected outcome is not obtained, an individual should be sensitive to not only whether the choice itself was suboptimal but also whether the action required to indicate that choice was executed successfully. The intelligent assignment of credit to action execution versus action selection has clear ecological utility for the learner. To explore this, we used a modified version of a classic reinforcement learning task in which feedback indicated whether negative prediction errors were, or were not, associated with execution errors. Using fMRI, we asked if prediction error computations in the human striatum, a key substrate in reinforcement learning and decision making, are modulated when a failure in action execution results in the negative outcome. Participants were more tolerant of non-rewarded outcomes when these resulted from execution errors versus when execution was successful, but reward was withheld. Consistent with this behavior, a model-driven analysis of neural activity revealed an attenuation of the signal associated with negative reward prediction errors in the striatum following execution failures. These results converge with other lines of evidence suggesting that prediction errors in the mesostriatal dopamine system integrate high-level information during the evaluation of instantaneous reward outcomes.

AB - Decisions must be implemented through actions, and actions are prone to error. As such, when an expected outcome is not obtained, an individual should be sensitive to not only whether the choice itself was suboptimal but also whether the action required to indicate that choice was executed successfully. The intelligent assignment of credit to action execution versus action selection has clear ecological utility for the learner. To explore this, we used a modified version of a classic reinforcement learning task in which feedback indicated whether negative prediction errors were, or were not, associated with execution errors. Using fMRI, we asked if prediction error computations in the human striatum, a key substrate in reinforcement learning and decision making, are modulated when a failure in action execution results in the negative outcome. Participants were more tolerant of non-rewarded outcomes when these resulted from execution errors versus when execution was successful, but reward was withheld. Consistent with this behavior, a model-driven analysis of neural activity revealed an attenuation of the signal associated with negative reward prediction errors in the striatum following execution failures. These results converge with other lines of evidence suggesting that prediction errors in the mesostriatal dopamine system integrate high-level information during the evaluation of instantaneous reward outcomes.

KW - decision making

KW - error

KW - reaching

KW - reinforcement learning

KW - striatum

UR - http://www.scopus.com/inward/record.url?scp=85065522881&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85065522881&partnerID=8YFLogxK

U2 - 10.1016/j.cub.2019.04.011

DO - 10.1016/j.cub.2019.04.011

M3 - Article

VL - 29

SP - 1606-1613.e5

JO - Current Biology

JF - Current Biology

SN - 0960-9822

IS - 10

ER -