Data augmentation for Convolutional LSTM based brain computer interface system

• Published in: Applied soft computing Vol. 122; p. 108811
• Main Authors: Takahashi, Kahoko, Sun, Zhe, Solé-Casals, Jordi, Cichocki, Andrzej, Phan, Anh Huy, Zhao, Qibin, Zhao, Hui-Hai, Deng, Shangkun, Micheletto, Ruggero
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.06.2022
• Subjects: Brain computer interface; Convolutional LSTM neural network; Data augmentation; Empirical Mode Decomposition; Empirical Mode Decomposition; Data augmentation; Brain computer interface; Convolutional LSTM neural network
• ISSN: 1568-4946; 1872-9681
• DOI: 10.1016/j.asoc.2022.108811

Electroencephalogram (EEG) is a noninvasive method to detect spatio-temporal electric signals in human brain, actively used in the recent development of Brain Computer Interfaces (BCI). EEG’s patterns are affected by the task, but also other variable factors influence the subject focus on the task and result in noisy EEG signals difficult to decipher. To surpass these limitations methods based on artificial neural networks (ANNs) are used, they are inherently robust to noise and do not require models. However, they learn from examples and require lots of training data-sets. This will increase costs, need research time and subjects effort. To reduce the number of experiments necessary for network training, we devised a methodology to provide artificial data from a limited number of training data-sets. This was done by applying Empirical Mode Decomposition (EMD) on the EEG frames and intermixing their Intrinsic Mode Function (IMFs). We experimented on motor imagery (MI) tests where participants were asked to imagine movement of the left (or right) arm while under EEG recording. The EEG data were firstly transformed using the Morlet wavelet and then fed to an originally designed Convolutional Neural Network (CNN) with long short term memory blocks (LSTM-RNN). The introduction of artificial frames improved performances when compared with standard algorithms. The artificial frames become advantageous even when the number of available real frames was only of 7 or 8. In a test with two subjects (200 recordings for each subject), we reached an accuracy better than 88% for both subjects. Improvements due to the artificial data were especially noticeable for the under-performing subject, whose EEG had lower accuracy. Imagination recognition accuracy was about 89% with 360 training frames, in which 300 were artificially created starting from 60 real ones. We believe this methodology of synthesizing artificial data may contribute to the development of novel and more efficient ways to train neural networks for brain computer interfaces. •A methodology was devised to generate artificial data from a limited number of training data-sets.•Pattern detection performance due to the artificial data were especially noticeable.•The approach used EMD on the EEG frames and intermixing their Intrinsic Mode Function.