TY - JOUR
T1 - Orbitofrontal Cortex and Learning Predictions of State Transitions
AU - Chan, Stephanie C.Y.
AU - Schuck, Nicolas W.
AU - Lopatina, Nina
AU - Schoenbaum, Geoffrey
AU - Niv, Yael
N1 - Funding Information:
This work was supported by the National Institute of Mental Health (R01MH0988861 and T32MH065214); a Sloan Research Fellowship (to Yael Niv); the National Science Foundation (Collaborative Research in Computational Neuroscience award IIS1009452); the U.S. Army Research Laboratory and the U.S. Army Research Office (W911NF1410101); the National Institute on Drug Abuse (Geoffrey Schoenbaum and Nina Lopatina), an Independent Max Planck Research Group grant awarded by the Max Planck Society (M.TN.A.BILD0004, Nicolas W. Schuck), and a Starting Grant from the European Union (ERC-2019-StG-REPLAY- 852669, Nicolas W. Schuck). The opinions expressed in this article are the authors’ own and do not reflect the view of the NIH/DHHS.
Publisher Copyright:
© 2021 American Psychological Association
PY - 2021
Y1 - 2021
N2 - The orbitofrontal cortex (OFC) has been implicated in goal-directed planning and model-based decisionmaking. One key prerequisite for model-based decision-making is learning the transition structure of the environment—the probabilities of transitioning from one environmental state to another. In this work, we investigated how the OFC might be involved in learning this transition structure, by using fMRI to assess OFC activity while humans experienced probabilistic cue–outcome transitions. We found that OFC activity was indeed correlated with behavioral measures of learning about transition structure. On a trial-by-trial basis, OFC activity was associated with subsequently increased expectation of the more probable outcome; that is, with subsequently more optimal cue–outcome predictions. Interestingly, this relationship was observed no matter what outcome occurred at the time of the OFC activity, and thus is inconsistent with an interpretation of the OFC activity as representing a “state prediction error” that would facilitate learning transitions via error-correcting mechanisms. Finally, OFC activity was related to more optimal predictions only for subsequent trials involving the same cue that was observed at the time of OFC activity—this relationship was not observed for subsequent trials involving a different cue. All together, these results indicate that the OFC is involved in updating or reinforcing a learned transition model on a trial-by-trial basis, specifically for the currently observed cue–outcome associations.
AB - The orbitofrontal cortex (OFC) has been implicated in goal-directed planning and model-based decisionmaking. One key prerequisite for model-based decision-making is learning the transition structure of the environment—the probabilities of transitioning from one environmental state to another. In this work, we investigated how the OFC might be involved in learning this transition structure, by using fMRI to assess OFC activity while humans experienced probabilistic cue–outcome transitions. We found that OFC activity was indeed correlated with behavioral measures of learning about transition structure. On a trial-by-trial basis, OFC activity was associated with subsequently increased expectation of the more probable outcome; that is, with subsequently more optimal cue–outcome predictions. Interestingly, this relationship was observed no matter what outcome occurred at the time of the OFC activity, and thus is inconsistent with an interpretation of the OFC activity as representing a “state prediction error” that would facilitate learning transitions via error-correcting mechanisms. Finally, OFC activity was related to more optimal predictions only for subsequent trials involving the same cue that was observed at the time of OFC activity—this relationship was not observed for subsequent trials involving a different cue. All together, these results indicate that the OFC is involved in updating or reinforcing a learned transition model on a trial-by-trial basis, specifically for the currently observed cue–outcome associations.
KW - Model-based decision making
KW - Orbitofrontal cortex
KW - Planning
KW - State transitions
UR - http://www.scopus.com/inward/record.url?scp=85116959232&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85116959232&partnerID=8YFLogxK
U2 - 10.1037/bne0000461
DO - 10.1037/bne0000461
M3 - Article
C2 - 34291969
AN - SCOPUS:85116959232
SN - 0735-7044
VL - 135
SP - 487
EP - 497
JO - Behavioral Neuroscience
JF - Behavioral Neuroscience
IS - 4
ER -