Dual-system learning models and drugs of abuse

Dylan A. Simon, Nathaniel D. Daw

Research output: Chapter in Book/Report/Conference proceedingChapter

10 Scopus citations

Abstract

Dual-system theories in psychology and neuroscience propose that a deliberative or goal-directed decision system is accompanied by a more automatic or habitual path to action. In computational terms, the latter is prominently associated with model-free reinforcement learning algorithms such as temporaldifference learning, and the former with model-based approaches. Due in part to the close association between drugs of abuse and dopamine, and also between dopamine, temporal-difference learning, and habitual behavior, addictive drugs are often thought to specifically target the habitual system. However, although many drug-taking behaviors are well explained under such a theory, evidence suggests that drug-seeking behaviors must leverage a goal-directed controller as well. Indeed, one exhaustive theoretical account proposed that drugs may have numerous, distinct impacts on both systems as well as on other processes. Here, we seek a more parsimonious account of these phenomena by asking whether the apparent profligacy of drugs' effects might be explained by a single mechanism of action. In particular, we propose that the pattern of effects observed under drug abuse may reveal interactions between the two controllers, which have typically been modeled as separate and parallel. We sketch several different candidate characterizations and architectures by which model-free effects may impinge on a model-based system, including sharing of cached values through truncated tree search and bias of transition selection for prioritized value sweeping.

Original languageEnglish (US)
Title of host publicationComputational Neuroscience of Drug Addiction
PublisherSpringer New York
Pages145-161
Number of pages17
ISBN (Electronic)9781461407515
ISBN (Print)9781461407508
DOIs
StatePublished - Jan 1 2012
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Medicine(all)
  • Neuroscience(all)

Fingerprint Dive into the research topics of 'Dual-system learning models and drugs of abuse'. Together they form a unique fingerprint.

Cite this