A simple model for learning in volatile environments

Payam Piray, Nathaniel D. Daw

Research output: Contribution to journalArticlepeer-review

29 Scopus citations


Sound principles of statistical inference dictate that uncertainty shapes learning. In this work, we revisit the question of learning in volatile environments, in which both the first and second-order statistics of observations dynamically evolve over time. We propose a new model, the volatile Kalman filter (VKF), which is based on a tractable state-space model of uncertainty and extends the Kalman filter algorithm to volatile environments. The proposed model is algorithmically simple and encompasses the Kalman filter as a special case. Specifically, in addition to the error-correcting rule of Kalman filter for learning observations, the VKF learns volatility according to a second error-correcting rule. These dual updates echo and contextualize classical psychological models of learning, in particular hybrid accounts of Pearce-Hall and Rescorla-Wagner. At the computational level, compared with existing models, the VKF gives up some flexibility in the generative model to enable a more faithful approximation to exact inference. When fit to empirical data, the VKF is better behaved than alternatives and better captures human choice data in two independent datasets of probabilistic learning tasks. The proposed model provides a coherent account of learning in stable or volatile environments and has implications for decision neuroscience research.

Original languageEnglish (US)
Article numbere1007963
JournalPLoS computational biology
Issue number7
StatePublished - Jul 2020

All Science Journal Classification (ASJC) codes

  • Genetics
  • Ecology, Evolution, Behavior and Systematics
  • Cellular and Molecular Neuroscience
  • Molecular Biology
  • Ecology
  • Computational Theory and Mathematics
  • Modeling and Simulation


Dive into the research topics of 'A simple model for learning in volatile environments'. Together they form a unique fingerprint.

Cite this