IFNet: An Interactive Frequency Convolutional Neural Network for Enhancing Motor Imagery Decoding From EEG

• Published in: IEEE transactions on neural systems and rehabilitation engineering Vol. 31; pp. 1900 - 1911
• Main Authors: Wang, Jiaheng, Yao, Lin, Wang, Yueming
• Format: Journal Article
• Language: English
• Published: United States IEEE 2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Artificial neural networks; Brain; Brain modeling; Brain-Computer Interfaces; Brain–computer interface; Classification; Computer applications; Convolution; Convolutional neural networks; Coupling; cross-frequency interactions; data augmentation; Datasets; Decision analysis; Decoding; EEG; Electroencephalography; Feature extraction; Frequencies; Human-computer interface; Humans; Image enhancement; Imagination - physiology; Implants; Mental task performance; motor imagery; Neural networks; Neural Networks, Computer; Sensitivity analysis; Task analysis
• ISSN: 1534-4320; 1558-0210; 1558-0210
• DOI: 10.1109/TNSRE.2023.3257319

Objective: The key principle of motor imagery (MI) decoding for electroencephalogram (EEG)-based Brain-Computer Interface (BCI) is to extract task-discriminative features from spectral, spatial, and temporal domains jointly and efficiently, whereas limited, noisy, and non-stationary EEG samples challenge the advanced design of decoding algorithms. Methods: Inspired by the concept of cross-frequency coupling and its correlation with different behavioral tasks, this paper proposes a lightweight Interactive Frequency Convolutional Neural Network (IFNet) to explore cross-frequency interactions for enhancing representation of MI characteristics. IFNet first extracts spectro-spatial features in low and high-frequency bands, respectively. Then the interplay between the two bands is learned using an element-wise addition operation followed by temporal average pooling. Combined with repeated trial augmentation as a regularizer, IFNet yields spectro-spatio-temporally robust features for the final MI classification. We conduct extensive experiments on two benchmark datasets: the BCI competition IV 2a (BCIC-IV-2a) dataset and the OpenBMI dataset. Results: Compared with state-of-the-art MI decoding algorithms, IFNet achieves significantly superior classification performance on both datasets while improving the winner's result in BCIC-IV-2a by 11%. Moreover, by conducting sensitivity analysis on decision windows, we show IFNet attains the best trade-off between decoding speed and accuracy. Detailed analysis and visualization verify IFNet can capture the coupling across frequency bands along with the known MI signatures. Conclusion: We demonstrate the effectiveness and superiority of the proposed IFNet for MI decoding. Significance: This study suggests IFNet holds promise for rapid response and accurate control in MI-BCI applications.