Common ERP responses to narrative incoherence in sentence and picture pair comprehension

• Published in: Brain and cognition Vol. 153; p. 105775
• Main Authors: Jouen, Anne-Lise, Cazin, Nicolas, Hidot, Sullivan, Madden-Lombardi, Carol, Ventre-Dominey, Jocelyne, Dominey, Peter Ford
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier Inc 01.10.2021; Elsevier Science; Elsevier
• Subjects: Cognitive ability; Cognitive science; EEG; ERP; Event-related potentials; Functional magnetic resonance imaging; Information processing; Language; Late positivity; Narrative; Nervous system; Semantics; Late positivity; ERP; Language; Narrative
• ISSN: 0278-2626; 1090-2147
• DOI: 10.1016/j.bandc.2021.105775

•Shared ERP response to narrative incoherence produced by short narratives in written language and visual images.•• Common ERP responses include late positivities previously associated independently with written language and visual images.•• All separate sentences and images are grammatically and visually correct and well formed. The ERP is in response to a narrative incoherence.•• Of interest in the perspective of narrative processing, and a common multimodal meaning representation system. Understanding the neural processes underlying the comprehension of visual images and sentences remains a major open challenge in cognitive neuroscience. We previously demonstrated with fMRI and DTI that comprehension of visual images and sentences describing human activities recruits a common extended parietal-temporal-frontal semantic system. The current research tests the hypothesis that this common semantic system will display similar ERP profiles during processing in these two modalities, providing further support for the common comprehension system. We recorded EEG from naïve subjects as they saw simple narratives made up of a first visual image depicting a human event, followed by a second image that was either a sequentially coherent narrative follow-up, or not, of the first. Incoherent second stimuli depict the same agents but shifted into a different situation. In separate blocks of trials the same protocol was presented using narrative sentence stimuli. Part of the novelty is the comparison of sentence and visual narrative responses. ERPs revealed common neural profiles for narrative processing across image and sentence modalities in the form of early and late central and frontal positivities in response to narrative incoherence. There was an additional posterior positivity only for sentences in a very late window. These results are discussed in the context of ERP signatures of narrative processing and meaning, and a current model of narrative comprehension.