Naturalistic Audiovisual Illusions Reveal the Cortical Sites Involved in the Multisensory Processing of Speech

• Published in: The European journal of neuroscience Vol. 61; no. 5; pp. e70043 - n/a
• Main Authors: Mégevand, Pierre, Thézé, Raphaël, Mehta, Ashesh D.
• Format: Journal Article
• Language: English
• Published: France Wiley Subscription Services, Inc 01.03.2025
• Subjects: Acoustic Stimulation; Adult; audiovisual speech; Auditory Cortex - physiology; Cerebral Cortex - physiology; comprehension; Electroencephalography; Female; Hearing; Humans; Illusions - physiology; intracranial electroencephalography; Male; multisensory processing; Photic Stimulation; Physical characteristics; Sensory integration; Sensory properties; Speech; Speech - physiology; speech and language; Speech perception; Speech Perception - physiology; Visual cortex; Visual Cortex - physiology; Visual perception; Visual Perception - physiology; Visual stimuli; Young Adult; intracranial electroencephalography; speech and language; audiovisual speech; multisensory processing; comprehension; humans
• ISSN: 0953-816X; 1460-9568; 1460-9568
• DOI: 10.1111/ejn.70043

ABSTRACT Audiovisual speech illusions are a spectacular illustration of the effect of visual cues on the perception of speech. Because they allow dissociating perception from the physical characteristics of the sensory inputs, these illusions are useful to investigate the cerebral processing of audiovisual speech. However, the meaningless, monosyllabic utterances typically used to induce illusions are far removed from natural communication through speech. We developed naturalistic speech stimuli that embed mismatched auditory and visual cues within grammatically correct sentences to induce illusory perceptions in controlled fashion. Using intracranial EEG, we confirmed that the cortical processing of audiovisual speech recruits an ensemble of areas, from auditory and visual cortices to multisensory and associative regions. Importantly, we were able to resolve which cortical areas are driven more by the auditory or the visual contents of the speech stimulus or by the eventual perceptual report. Our results suggest that higher order sensory and associative areas, rather than early sensory cortices, are key loci for illusory perception. Naturalistic audiovisual speech illusions represent a powerful tool to dissect the specific roles of individual cortical areas in the processing of audiovisual speech. We designed innovative audiovisual speech illusions and used intracranial EEG to localize the cortical sites of multisensory speech processing. Whereas sites that tracked the auditory and visual components of speech stimuli concentrated in the respective sensory cortices, sites that were sensitive to the interaction between the auditory and visual components of speech were more distributed in higher order cortical areas.