Credit assignment in movement-dependent reinforcement learning

Samuel D. McDougle, Matthew J. Boggess, Matthew J. Crossley, Darius Parvin, Richard B. Ivry, Jordan A. Taylor

Research output: Contribution to journalArticlepeer-review

38 Scopus citations


When a person fails to obtain an expected reward from an object in the environment, they face a credit assignment problem: Did the absence of reward reflect an extrinsic property of the environment or an intrinsic error in motor execution? To explore this problem, we modified a popular decision-making task used in studies of reinforcement learning, the two-armed bandit task. We compared a version in which choices were indicated by key presses, the standard response in such tasks, to a version in which the choices were indicated by reaching movements, which affords execution failures. In the key press condition, participants exhibited a strong risk aversion bias; strikingly, this bias reversed in the reaching condition. This result can be explained by a reinforcement model whereinmovement errors influence decision-making, either by gating reward prediction errors or by modifying an implicit representation of motor competence. Two further experiments support the gating hypothesis. First, we used a condition in which we provided visual cues indicative of movement errors but informed the participants that trial outcomes were independent of their actual movements. The main result was replicated, indicating that the gating process is independent of participants' explicit sense of control. Second, individuals with cerebellar degeneration failed to modulate their behavior between the key press and reach conditions, providing converging evidence of an implicit influence of movement error signals on reinforcement learning. These results provide a mechanistically tractable solution to the credit assignment problem.

Original languageEnglish (US)
Pages (from-to)6797-6802
Number of pages6
JournalProceedings of the National Academy of Sciences of the United States of America
Issue number24
StatePublished - Jun 14 2016

All Science Journal Classification (ASJC) codes

  • General


  • Cerebellum
  • Decision-making
  • Reinforcement learning
  • Reward prediction error
  • Sensory prediction error


Dive into the research topics of 'Credit assignment in movement-dependent reinforcement learning'. Together they form a unique fingerprint.

Cite this