Learning EEG Motor Characteristics via Temporal-Spatial Representations

• Published in: IEEE transactions on emerging topics in computational intelligence Vol. 9; no. 1; pp. 933 - 945
• Main Authors: Xiang, Tian-Yu, Zhou, Xiao-Hu, Xie, Xiao-Liang, Liu, Shi-Qi, Yang, Hong-Jun, Feng, Zhen-Qiu, Gui, Mei-Jiang, Li, Hao, Huang, De-Xing, Liu, Xiu-Ling, Hou, Zeng-Guang
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.02.2025; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; Brain modeling; CNN-GNN structure; Electrodes; Electroencephalogram; Electroencephalography; Feature extraction; Graph neural networks; Graphical representations; Human-computer interface; Imaging techniques; Learning; motor characteristics; Motors; Multisensor fusion; Neural networks; Spatial data; Spatiotemporal data; Task analysis; temporal-spatial representations; Time-frequency analysis
• ISSN: 2471-285X; 2471-285X
• DOI: 10.1109/TETCI.2024.3425328

Electroencephalogram (EEG) is a widely used neural imaging technique for modeling motor characteristics. However, current studies have primarily focused on temporal representations of EEG, with less emphasis on the spatial and functional connections among electrodes. This study introduces a novel two-stream model to analyze both temporal and spatial representations of EEG for learning motor characteristics. Temporal representations are extracted with a set of convolutional neural networks (CNN) treated as dynamic filters, while spatial representations are learned by graph neural networks (GNN) using learnable adjacency matrices. At each stage, a res-block is designed to integrate temporal and spatial representations, facilitating a fusion of temporal-spatial information. Finally, the summarized representations of both streams are fused with fully connected neural networks to learn motor characteristics. Experimental evaluations on the Physionet, OpenBMI, and BCI Competition IV Dataset 2a demonstrate the model's efficacy, achieving accuracies of <inline-formula><tex-math notation="LaTeX">73.6\%/70.4\%</tex-math></inline-formula> for four-class subject-dependent/independent paradigms, <inline-formula><tex-math notation="LaTeX">84.2\%/82.0\%</tex-math></inline-formula> for two-class subject-dependent/independent paradigms, and 78.5% for a four-class subject-dependent paradigm, respectively. The encouraged results underscore the model's potential in understanding EEG-based motor characteristics, paving the way for advanced brain-computer interface systems.