Dopamine, uncertainty and TD learning

Yael Niv, Michael O. Duff, Peter D. Dayan

Research output: Contribution to journalArticle

91 Scopus citations

Abstract

Substantial evidence suggests that the phasic activities of dopaminergic neurons in the primate midbrain represent a temporal difference (TD) error in predictions of future reward, with increases above and decreases below baseline consequent on positive and negative prediction errors, respectively. However, dopamine cells have very low baseline activity, which implies that the representation of these two sorts of error is asymmetric. We explore the implications of this seemingly innocuous asymmetry for the interpretation of dopaminergic firing patterns in experiments with probabilistic rewards which bring about persistent prediction errors. In particular, we show that when averaging the non-stationary prediction errors across trials, a ramping in the activity of the dopamine neurons should be apparent, whose magnitude is dependent on the learning rate. This exact phenomenon was observed in a recent experiment, though being interpreted there in antipodal terms as a within-trial encoding of uncertainty.

Original languageEnglish (US)
Article number6
JournalBehavioral and Brain Functions
Volume1
DOIs
StatePublished - May 4 2005
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Cognitive Neuroscience
  • Biological Psychiatry
  • Behavioral Neuroscience

Fingerprint Dive into the research topics of 'Dopamine, uncertainty and TD learning'. Together they form a unique fingerprint.

  • Cite this