Humans combine value learning and hypothesis testing strategically in multidimensional probabilistic reward learning

Mingyu Song, Persis A. Baah, Ming Bo Cai, Yael Niv

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

Realistic and complex decision tasks often allow for many possible solutions. How do we find the correct one? Introspection suggests a process of trying out solutions one after the other until success. However, such methodical serial testing may be too slow, especially in environments with noisy feedback. Alternatively, the underlying learning process may involve implicit reinforcement learning that learns about many possibilities in parallel. Here we designed a multi-dimensional probabilistic active-learning task tailored to study how people learn to solve such complex problems. Participants configured three-dimensional stimuli by selecting features for each dimension and received probabilistic reward feedback. We manipulated task complexity by changing how many feature dimensions were relevant to maximizing reward, as well as whether this information was provided to the participants. To investigate how participants learn the task, we examined models of serial hypothesis testing, feature-based reinforcement learning, and combinations of the two strategies. Model comparison revealed evidence for hypothesis testing that relies on reinforcement-learning when selecting what hypothesis to test. The extent to which participants engaged in hypothesis testing depended on the instructed task complexity: people tended to serially test hypotheses when instructed that there were fewer relevant dimensions, and relied more on gradual and parallel learning of feature values when the task was more complex. This demonstrates a strategic use of task information to balance the costs and benefits of the two methods of learning.

Original languageEnglish (US)
Article numbere1010699
JournalPLoS computational biology
Volume18
Issue number11
DOIs
StatePublished - Nov 2022

All Science Journal Classification (ASJC) codes

  • Genetics
  • Ecology, Evolution, Behavior and Systematics
  • Cellular and Molecular Neuroscience
  • Molecular Biology
  • Ecology
  • Computational Theory and Mathematics
  • Modeling and Simulation

Fingerprint

Dive into the research topics of 'Humans combine value learning and hypothesis testing strategically in multidimensional probabilistic reward learning'. Together they form a unique fingerprint.

Cite this