Breaking down the cocktail party: Attentional modulation of cerebral audiovisual speech processing

• Published in: NeuroImage (Orlando, Fla.) Vol. 224; p. 117365
• Main Authors: Wikman, Patrik, Sahari, Elisa, Salmela, Viljami, Leminen, Alina, Leminen, Miika, Laine, Matti, Alho, Kimmo
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 01.01.2021; Elsevier Limited; Elsevier
• Subjects: Acoustic Stimulation - methods; Adult; Attention task; Audiovisual speech; Auditory Cortex - physiology; Auditory Perception - physiology; Brain - physiology; Brain Mapping - methods; Brain research; Cocktail party; Cognitive ability; Cortex (auditory); Electroencephalography; Female; Females; fMRI; Hearing; Humans; Information processing; Listening; Magnetic resonance imaging; Magnetic Resonance Imaging - methods; Male; Multivariate pattern analysis (MVPA); Neural networks; Neuroimaging; Noise; Selective attention; Semantics; Sensory integration; Somatosensory cortex; Sound; Speech; Speech - physiology; Speech Perception - physiology; Visual Perception - physiology; Young Adult; Cocktail party; Selective attention; fMRI; Multivariate pattern analysis (MVPA); Semantics; Audiovisual speech
• ISSN: 1053-8119; 1095-9572
• DOI: 10.1016/j.neuroimage.2020.117365

•fMRI and multivariate pattern analysis elucidate attention to lifelike AV speech.•Attention in cocktail-party settings strongly modulates cortical activity patterns•Semantics affect attention-related activity patterns in primary auditory cortex.•Dynamics of attention to speech are context-dependent. Recent studies utilizing electrophysiological speech envelope reconstruction have sparked renewed interest in the cocktail party effect by showing that auditory neurons entrain to selectively attended speech. Yet, the neural networks of attention to speech in naturalistic audiovisual settings with multiple sound sources remain poorly understood. We collected functional brain imaging data while participants viewed audiovisual video clips of lifelike dialogues with concurrent distracting speech in the background. Dialogues were presented in a full-factorial design, comprising task (listen to the dialogues vs. ignore them), audiovisual quality and semantic predictability. We used univariate analyses in combination with multivariate pattern analysis (MVPA) to study modulations of brain activity related to attentive processing of audiovisual speech. We found attentive speech processing to cause distinct spatiotemporal modulation profiles in distributed cortical areas including sensory and frontal-control networks. Semantic coherence modulated attention-related activation patterns in the earliest stages of auditory cortical processing, suggesting that the auditory cortex is involved in high-level speech processing. Our results corroborate views that emphasize the dynamic nature of attention, with task-specificity and context as cornerstones of the underlying neuro-cognitive mechanisms.