Filter Bank Convolutional Neural Network for Short Time-Window Steady-State Visual Evoked Potential Classification

• Published in: IEEE transactions on neural systems and rehabilitation engineering Vol. 29; pp. 2615 - 2624
• Main Authors: Ding, Wenlong, Shan, Jianhua, Fang, Bin, Wang, Chengyin, Sun, Fuchun, Li, Xinyue
• Format: Journal Article
• Language: English
• Published: United States IEEE 2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Artificial neural networks; Asynchronous brain-computer interface (BCI) system; Brain; Brain-Computer Interfaces; Canonical Correlation Analysis; Computer applications; convolutional neural network (CNN); Convolutional neural networks; Correlation analysis; cross-individual; Datasets; Deep learning; Electrodes; Electroencephalography; Evoked Potentials, Visual; Fast Fourier transformations; Feature extraction; Filter banks; Fourier analysis; Fourier transforms; Frequency domain analysis; Harmonic analysis; Human-computer interface; Humans; Implants; inter-individual; Machine learning; Neural networks; Neural Networks, Computer; Reproducibility of Results; short time-window; Signal to noise ratio; Steady state; steady-state visual evoked potential (SSVEP); Time domain analysis; Training; Visual evoked potentials; Visualization
• ISSN: 1534-4320; 1558-0210; 1558-0210
• DOI: 10.1109/TNSRE.2021.3132162

Convolutional neural network (CNN) has been gradually applied to steady-state visual evoked potential (SSVEP) of the brain-computer interface (BCI). Frequency-domain features extracted by fast Fourier Transform (FFT) or time-domain signals are used as network input. In the frequency-domain diagram, the features at the short time-window are not obvious and the phase information of each electrode channel may be ignored as well. Hence we propose a time-domain-based CNN method (tCNN), using the time-domain signal as network input. And the filter bank tCNN (FB-tCNN) is further proposed to improve its performance in the short time-window. We compare FB-tCNN with the canonical correlation analysis (CCA) methods and other CNN methods in our dataset and public dataset. And FB-tCNN shows superior performance at the short time-window in the intra-individual test. At the 0.2 s time-window, the accuracy of our method reaches <inline-formula> <tex-math notation="LaTeX">88.36\, \pm \,4.89 </tex-math></inline-formula>% in our dataset, <inline-formula> <tex-math notation="LaTeX">77.78\, \pm \,2.16 </tex-math></inline-formula>% and <inline-formula> <tex-math notation="LaTeX">79.21\, \pm \,1.80 </tex-math></inline-formula>% respectively in the two sessions of the public dataset, which is higher than other methods. The impacts of training-subject number and data length in inter-individual or cross-individual are studied. FB-tCNN shows the potential in implementing inter-individual BCI. Further analysis shows that the deep learning method is easier in terms of the implementation of the asynchronous BCI system than the training data-driven CCA. The code is available for reproducibility at https://github.com/DingWenl/FB-tCNN .