TY - JOUR
T1 - Representation of real-world event schemas during narrative perception
AU - Baldassano, Christopher
AU - Hasson, Uri
AU - Norman, Kenneth A.
N1 - Funding Information:
Received Jan. 29, 2018; revised Sept. 12, 2018; accepted Sept. 12, 2018. Authorcontributions:C.B.wrotethefirstdraftofthepaper;C.B.,U.H.,andK.A.N.editedthepaper;C.B.,U.H.,and K.A.N. designed research; C.B. performed research; C.B. analyzed data; C.B. wrote the paper. This work was supported by Intel Labs to C.B. and National Institutes of Health Grant R01 MH112357-01 to U.H. and K.A.N. We thank R. Masís-Obando for input on the Discussion; and the members of the U.H. and K.A.N. laboratories for comments and support. The authors declare no competing financial interests. Correspondence should be addressed to Dr. Christopher Baldassano, Columbia University, 1190 Amsterdam Avenue, New York, NY 10027. E-mail: c.baldassano@columbia.edu. https://doi.org/10.1523/JNEUROSCI.0251-18.2018 Copyright © 2018 the authors 0270-6474/18/389689-11$15.00/0
Publisher Copyright:
© 2018, Society for Neuroscience. All rights reserved.
PY - 2018/11/7
Y1 - 2018/11/7
N2 - Understanding movies and stories requires maintaining a high-level situation model that abstracts away from perceptual details to describe the location, characters, actions, and causal relationships of the currently unfolding event. These models are built not only from information present in the current narrative, but also from prior knowledge about schematic event scripts, which describe typical event sequences encountered throughout a lifetime. We analyzed fMRI data from 44 human subjects (male and female) presented with 16 three-minute stories, consisting of four schematic events drawn from two different scripts (eating at a restaurant or going through the airport). Aside from this shared script structure, the stories varied widely in terms of their characters and storylines, and were presented in two highly dissimilar formats (audiovisual clips or spoken narration). One group was presented with the stories in an intact temporal sequence, while a separate control group was presented with the same events in scrambled order. Regions including the posterior medial cortex, medial prefrontal cortex (mPFC), and superior frontal gyrus exhibited schematic event patterns that generalized across stories, subjects, and modalities. Patterns in mPFC were also sensitive to overall script structure, with temporally scrambled events evoking weaker schematic representations. Using a Hidden Markov Model, patterns in these regions predicted the script (restaurant vs airport) of unlabeled data with high accuracy and were used to temporally align multiple stories with a shared script. These results extend work on the perception of controlled, artificial schemas in human and animal experiments to naturalistic perception of complex narratives.
AB - Understanding movies and stories requires maintaining a high-level situation model that abstracts away from perceptual details to describe the location, characters, actions, and causal relationships of the currently unfolding event. These models are built not only from information present in the current narrative, but also from prior knowledge about schematic event scripts, which describe typical event sequences encountered throughout a lifetime. We analyzed fMRI data from 44 human subjects (male and female) presented with 16 three-minute stories, consisting of four schematic events drawn from two different scripts (eating at a restaurant or going through the airport). Aside from this shared script structure, the stories varied widely in terms of their characters and storylines, and were presented in two highly dissimilar formats (audiovisual clips or spoken narration). One group was presented with the stories in an intact temporal sequence, while a separate control group was presented with the same events in scrambled order. Regions including the posterior medial cortex, medial prefrontal cortex (mPFC), and superior frontal gyrus exhibited schematic event patterns that generalized across stories, subjects, and modalities. Patterns in mPFC were also sensitive to overall script structure, with temporally scrambled events evoking weaker schematic representations. Using a Hidden Markov Model, patterns in these regions predicted the script (restaurant vs airport) of unlabeled data with high accuracy and were used to temporally align multiple stories with a shared script. These results extend work on the perception of controlled, artificial schemas in human and animal experiments to naturalistic perception of complex narratives.
KW - Event
KW - FMRI
KW - Narrative
KW - Perception
KW - Schema
KW - Script
UR - http://www.scopus.com/inward/record.url?scp=85056426982&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85056426982&partnerID=8YFLogxK
U2 - 10.1523/JNEUROSCI.0251-18.2018
DO - 10.1523/JNEUROSCI.0251-18.2018
M3 - Article
C2 - 30249790
AN - SCOPUS:85056426982
SN - 0270-6474
VL - 38
SP - 9689
EP - 9699
JO - Journal of Neuroscience
JF - Journal of Neuroscience
IS - 45
ER -