Explicit melioration by a neural diffusion model

Patrick Simen, Jonathan D. Cohen

Research output: Contribution to journalArticle

21 Scopus citations

Abstract

When faced with choices between two sources of reward, animals can rapidly adjust their rates of responding to each so that overall reinforcement increases. Herrnstein's 'matching law' provides a simple description of the equilibrium state of this choice allocation process: animals reallocate behavior so that relative rates of responding equal, or match, the relative rates of reinforcement obtained for each response. Herrnstein and colleagues proposed 'melioration' as a dynamical process for achieving this equilibrium, but left details of its operation unspecified. Here we examine a way of filling in the details that links the decision making and operant conditioning literatures and extends choice proportion predictions into predictions about inter-response times. Our approach implements melioration in an adaptive version of the drift diffusion model (DDM), which is widely used in decision making research to account for response time distributions. When the drift parameter of the DDM is 0 and its threshold parameters are inversely proportional to reward rates, its choice proportions dynamically track a state of exact matching. A DDM with fixed thresholds and drift that is determined by differences in reward rates can produce similar, but not identical, results. We examine the choice probability and inter-response time predictions of these models, separately and in combination, and the possible implications for brain organization provided by neural network implementations of them. Results suggest that melioration and matching may derive from synapses that estimate reward rates by a process of leaky integration, and that link together the input and output stages of a two-stage stimulus-response mechanism.

Original languageEnglish (US)
Pages (from-to)95-117
Number of pages23
JournalBrain Research
Volume1299
DOIs
StatePublished - Nov 3 2009

All Science Journal Classification (ASJC) codes

  • Neuroscience(all)
  • Molecular Biology
  • Clinical Neurology
  • Developmental Biology

Keywords

  • Decision making
  • Drift diffusion
  • Instrumental conditioning
  • Matching
  • Melioration
  • Neural network
  • Operant conditioning
  • Reinforcement learning
  • Reward rate

Fingerprint Dive into the research topics of 'Explicit melioration by a neural diffusion model'. Together they form a unique fingerprint.

  • Cite this