Speech decoding using cortical and subcortical electrophysiological signals

• Published in: Frontiers in neuroscience Vol. 18; p. 1345308
• Main Authors: Wu, Hemmings, Cai, Chengwei, Ming, Wenjie, Chen, Wangyu, Zhu, Zhoule, Feng, Chen, Jiang, Hongjie, Zheng, Zhe, Sawan, Mohamad, Wang, Ting, Zhu, Junming
• Format: Journal Article
• Language: English
• Published: Switzerland Frontiers Media S.A 29.02.2024
• Subjects: decoding; machine learning; neural network; Neuroscience; sEEG; speech; sEEG; machine learning; speech; neural network; decoding
• ISSN: 1662-4548; 1662-453X; 1662-453X
• DOI: 10.3389/fnins.2024.1345308

Language impairments often result from severe neurological disorders, driving the development of neural prosthetics utilizing electrophysiological signals to restore comprehensible language. Previous decoding efforts primarily focused on signals from the cerebral cortex, neglecting subcortical brain structures' potential contributions to speech decoding in brain-computer interfaces. In this study, stereotactic electroencephalography (sEEG) was employed to investigate subcortical structures' role in speech decoding. Two native Mandarin Chinese speakers, undergoing sEEG implantation for epilepsy treatment, participated. Participants read Chinese text, with 1-30, 30-70, and 70-150 Hz frequency band powers of sEEG signals extracted as key features. A deep learning model based on long short-term memory assessed the contribution of different brain structures to speech decoding, predicting consonant articulatory place, manner, and tone within single syllable. Cortical signals excelled in articulatory place prediction (86.5% accuracy), while cortical and subcortical signals performed similarly for articulatory manner (51.5% vs. 51.7% accuracy). Subcortical signals provided superior tone prediction (58.3% accuracy). The superior temporal gyrus was consistently relevant in speech decoding for consonants and tone. Combining cortical and subcortical inputs yielded the highest prediction accuracy, especially for tone. This study underscores the essential roles of both cortical and subcortical structures in different aspects of speech decoding.