Increased Connectivity among Sensory and Motor Regions during Visual and Audiovisual Speech Perception

• Published in: The Journal of neuroscience Vol. 42; no. 3; pp. 435 - 442
• Main Authors: Peelle, Jonathan E, Spehar, Brent, Jones, Michael S, McConkey, Sarah, Myerson, Joel, Hale, Sandra, Sommers, Mitchell S, Tye-Murray, Nancy
• Format: Journal Article
• Language: English
• Published: United States Society for Neuroscience 19.01.2022
• Subjects: Adult; Aged; Aged, 80 and over; Auditory Cortex - diagnostic imaging; Auditory Cortex - physiology; Background noise; Brain; Brain mapping; Cortex (auditory); Cortex (premotor); Cortex (somatosensory); Cortex (temporal); Female; Functional magnetic resonance imaging; Hearing; Humans; Language; Lipreading; Magnetic Resonance Imaging; Male; Middle Aged; Nerve Net - diagnostic imaging; Nerve Net - physiology; Neural networks; Noise; Perception; Prefrontal cortex; Sensorimotor integration; Sensory integration; Speech; Speech perception; Speech Perception - physiology; Superior temporal sulcus; Synchronism; Synchronization; Visual cortex; Visual Cortex - diagnostic imaging; Visual Cortex - physiology; Visual perception; Visual Perception - physiology; Visual signals; Young Adult; language; speech; speechreading; audiovisual integration; lipreading
• ISSN: 0270-6474; 1529-2401
• DOI: 10.1523/JNEUROSCI.0114-21.2021

In everyday conversation, we usually process the talker's face as well as the sound of the talker's voice. Access to visual speech information is particularly useful when the auditory signal is degraded. Here, we used fMRI to monitor brain activity while adult humans ( = 60) were presented with visual-only, auditory-only, and audiovisual words. The audiovisual words were presented in quiet and in several signal-to-noise ratios. As expected, audiovisual speech perception recruited both auditory and visual cortex, with some evidence for increased recruitment of premotor cortex in some conditions (including in substantial background noise). We then investigated neural connectivity using psychophysiological interaction analysis with seed regions in both primary auditory cortex and primary visual cortex. Connectivity between auditory and visual cortices was stronger in audiovisual conditions than in unimodal conditions, including a wide network of regions in posterior temporal cortex and prefrontal cortex. In addition to whole-brain analyses, we also conducted a region-of-interest analysis on the left posterior superior temporal sulcus (pSTS), implicated in many previous studies of audiovisual speech perception. We found evidence for both activity and effective connectivity in pSTS for visual-only and audiovisual speech, although these were not significant in whole-brain analyses. Together, our results suggest a prominent role for cross-region synchronization in understanding both visual-only and audiovisual speech that complements activity in integrative brain regions like pSTS. In everyday conversation, we usually process the talker's face as well as the sound of the talker's voice. Access to visual speech information is particularly useful when the auditory signal is hard to understand (e.g., background noise). Prior work has suggested that specialized regions of the brain may play a critical role in integrating information from visual and auditory speech. Here, we show a complementary mechanism relying on synchronized brain activity among sensory and motor regions may also play a critical role. These findings encourage reconceptualizing audiovisual integration in the context of coordinated network activity.