A Temporal Dependency Learning CNN With Attention Mechanism for MI-EEG Decoding

• Published in: IEEE transactions on neural systems and rehabilitation engineering Vol. 31; pp. 3188 - 3200
• Main Authors: Ma, Xinzhi, Chen, Weihai, Pei, Zhongcai, Liu, Jingmeng, Huang, Bin, Chen, Jianer
• Format: Journal Article
• Language: English
• Published: United States IEEE 2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Artificial neural networks; attention mechanism; Brain; Brain–computer interface (BCI); Computer applications; Convolution; convolutional neural networks (CNNs); Data mining; Datasets; Decoding; Deep learning; EEG; Electroencephalography; Feature extraction; Human-computer interface; Implants; Machine learning; Mental task performance; motor imagery (MI); Neural networks; Spatial data; Task analysis; temporal dependency learning; Windows (intervals)
• ISSN: 1534-4320; 1558-0210
• DOI: 10.1109/TNSRE.2023.3299355

Deep learning methods have been widely explored in motor imagery (MI)-based brain computer interface (BCI) systems to decode electroencephalography (EEG) signals. However, most studies fail to fully explore temporal dependencies among MI-related patterns generated in different stages during MI tasks, resulting in limited MI-EEG decoding performance. Apart from feature extraction, learning temporal dependencies is equally important to develop a subject-specific MI-based BCI because every subject has their own way of performing MI tasks. In this paper, a novel temporal dependency learning convolutional neural network (CNN) with attention mechanism is proposed to address MI-EEG decoding. The network first learns spatial and spectral information from multi-view EEG data via the spatial convolution block. Then, a series of non-overlapped time windows is employed to segment the output data, and the discriminative feature is further extracted from each time window to capture MI-related patterns generated in different stages. Furthermore, to explore temporal dependencies among discriminative features in different time windows, we design a temporal attention module that assigns different weights to features in various time windows and fuses them into more discriminative features. The experimental results on the BCI Competition IV-2a (BCIC-IV-2a) and OpenBMI datasets show that our proposed network outperforms the state-of-the-art algorithms and achieves the average accuracy of 79.48%, improved by 2.30% on the BCIC-IV-2a dataset. We demonstrate that learning temporal dependencies effectively improves MI-EEG decoding performance. The code is available at https://github.com/Ma-Xinzhi/LightConvNet