Learning joint space–time–frequency features for EEG decoding on small labeled data

• Published in: Neural networks Vol. 114; pp. 67 - 77
• Main Authors: Zhao, Dongye, Tang, Fengzhen, Si, Bailu, Feng, Xisheng
• Format: Journal Article
• Language: English
• Published: United States Elsevier Ltd 01.06.2019
• Subjects: Brain–computer interfaces; Convolutional neural network; Joint space–time–frequency feature learning; Small labeled data; Subject-to-subject weight transfer; Small labeled data; Joint space–time–frequency feature learning; Brain–computer interfaces; Convolutional neural network; Subject-to-subject weight transfer
• ISSN: 0893-6080; 1879-2782; 1879-2782
• DOI: 10.1016/j.neunet.2019.02.009

Brain–computer interfaces (BCIs), which control external equipment using cerebral activity, have received considerable attention recently. Translating brain activities measured by electroencephalography (EEG) into correct control commands is a critical problem in this field. Most existing EEG decoding methods separate feature extraction from classification and thus are not robust across different BCI users. In this paper, we propose to learn subject-specific features jointly with the classification rule. We develop a deep convolutional network (ConvNet) to decode EEG signals end-to-end by stacking time–frequency transformation, spatial filtering, and classification together. Our proposed ConvNet implements a joint space–time–frequency feature extraction scheme for EEG decoding. Morlet wavelet-like kernels used in our network significantly reduce the number of parameters compared with classical convolutional kernels and endow the features learned at the corresponding layer with a clear interpretation, i.e. spectral amplitude. We further utilize subject-to-subject weight transfer, which uses parameters of the networks trained for existing subjects to initialize the network for a new subject, to solve the dilemma between a large number of demanded data for training deep ConvNets and small labeled data collected in BCIs. The proposed approach is evaluated on three public data sets, obtaining superior classification performance compared with the state-of-the-art methods.