An exploration-exploitation model based on norepinepherine and dopamine activity

Samuel M. McClure, Mark S. Gilzenrat, Jonathan D. Cohen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

45 Scopus citations

Abstract

We propose a model by which dopamine (DA) and norepinepherine (NE) combine to alternate behavior between relatively exploratory and exploitative modes. The model is developed for a target detection task for which there is extant single neuron recording data available from locus coeruleus (LC) NE neurons. An exploration-exploitation trade-off is elicited by regularly switching which of the two stimuli are rewarded. DA functions within the model to change synaptic weights according to a reinforcement learning algorithm. Exploration is mediated by the state of LC firing, with higher tonic and lower phasic activity producing greater response variability. The opposite state of LC function, with lower baseline firing rate and greater phasic responses, favors exploitative behavior. Changes in LC firing mode result from combined measures of response conflict and reward rate, where response conflict is monitored using models of anterior cingulate cortex (ACC). Increased long-term response conflict and decreased reward rate, which occurs following reward contingency switch, favors the higher tonic state of LC function and NE release. This increases exploration, and facilitates discovery of the new target.

Original languageEnglish (US)
Title of host publicationAdvances in Neural Information Processing Systems 18 - Proceedings of the 2005 Conference
Pages867-874
Number of pages8
StatePublished - Dec 1 2005
Event2005 Annual Conference on Neural Information Processing Systems, NIPS 2005 - Vancouver, BC, Canada
Duration: Dec 5 2005Dec 8 2005

Publication series

NameAdvances in Neural Information Processing Systems
ISSN (Print)1049-5258

Other

Other2005 Annual Conference on Neural Information Processing Systems, NIPS 2005
CountryCanada
CityVancouver, BC
Period12/5/0512/8/05

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Information Systems
  • Signal Processing

Fingerprint Dive into the research topics of 'An exploration-exploitation model based on norepinepherine and dopamine activity'. Together they form a unique fingerprint.

  • Cite this

    McClure, S. M., Gilzenrat, M. S., & Cohen, J. D. (2005). An exploration-exploitation model based on norepinepherine and dopamine activity. In Advances in Neural Information Processing Systems 18 - Proceedings of the 2005 Conference (pp. 867-874). (Advances in Neural Information Processing Systems).