DRS-Net: A spatial–temporal affective computing model based on multichannel EEG data

• Published in: Biomedical signal processing and control Vol. 76; p. 103660
• Main Authors: Li, Jingjing, Wu, Xia, Zhang, Yumei, Yang, Honghong, Wu, Xiaojun
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.07.2022
• Subjects: Affective computing; DRS-Net; Long-short term memory; Multichannel EEG; Reservoir computing; Affective computing; DRS-Net; Multichannel EEG; Long-short term memory; Reservoir computing
• ISSN: 1746-8094; 1746-8108
• DOI: 10.1016/j.bspc.2022.103660

•Provide DRS-Net, an end-to-end affective computing model using multichannel EEG data.•Automatically extract the EEG data’ spatial–temporal features with a dynamic reservoir state encoder.•Integrating Reservoir Computing with the neural network to deal with EEG data processing. Affective computing based on electroencephalography (EEG) is a promising field that highly integrates research and technology. A critical challenge is effectively extracting and integrating the temporal and spatial information to form a better representation for multichannel EEG data. Most existing studies use hand-selected features from each channel, which neglect high-dimensional dynamic temporal features and interplay of data from different electrodes. This study proposed a Dynamic Reservoir State Network (DRS-Net) to recognize the subject’s emotional states. The novel end-to-end model constructs a dynamic reservoir state encoder to extract multi-channel EEG data’s dynamic high dimension non-linear spatial–temporal information with high speed and low complexity. Then, a Long-Short Term Memory-dense decoder model is devised to detect emotional states. The effectiveness of the proposed DRS-Net model was evaluated on SEED, SEED-IV, and DEAP datasets. To validate the performance of the proposed method, we first combined the hand-selected features (differential entropy, power spectra density, fractal dimension, and statistics features) and classic machine learning classifiers methods (support vector machine, random forest, and k-nearest neighbor). Then, we compare them with the proposed method and other state-of-the-art deep learning methods. The experimental results generated by our method outperform all other methods in terms of accuracy and F1 score.