Multi-Hierarchical Fusion to Capture the Latent Invariance for Calibration-Free Brain-Computer Interfaces

• Published in: Frontiers in neuroscience Vol. 16; p. 824471
• Main Authors: Yang, Jun, Liu, Lintao, Yu, Huijuan, Ma, Zhengmin, Shen, Tao
• Format: Journal Article
• Language: English
• Published: Switzerland Frontiers Media S.A 25.04.2022
• Subjects: bidirectional long short-term memory; brain-computer interfaces; convolutional neural network; deep learning; motor imagery; Neuroscience; deep learning; motor imagery; brain-computer interfaces; bidirectional long short-term memory; convolutional neural network
• ISSN: 1662-4548; 1662-453X; 1662-453X
• DOI: 10.3389/fnins.2022.824471

Brain-computer interfaces (BCI) based motor imagery (MI) has become a research hotspot for establishing a flexible communication channel for patients with apoplexy or degenerative pathologies. Accurate decoding of motor imagery electroencephalography (MI-EEG) signals, while essential for effective BCI systems, is still challenging due to the significant noise inherent in the EEG signals and the lack of informative correlation between the signals and brain activities. The application of deep learning for EEG feature representation has been rarely investigated, nevertheless bringing improvements to the performance of motor imagery classification. This paper proposes a deep learning decoding method based on multi-hierarchical representation fusion (MHRF) on MI-EEG. It consists of a concurrent framework constructed of bidirectional LSTM (Bi-LSTM) and convolutional neural network (CNN) to fully capture the contextual correlations of MI-EEG and the spectral feature. Also, the stacked sparse autoencoder (SSAE) is employed to concentrate these two domain features into a high-level representation for cross-session and subject training guidance. The experimental analysis demonstrated the efficacy and practicality of the proposed approach using a public dataset from BCI competition IV and a private one collected by our MI task. The proposed approach can serve as a robust and competitive method to improve inter-session and inter-subject transferability, adding anticipation and prospective thoughts to the practical implementation of a calibration-free BCI system.