EEG classification model for virtual reality motion sickness based on multi-scale CNN feature correlation

• Published in: Computer methods and programs in biomedicine Vol. 251; p. 108218
• Main Authors: Hua, Chengcheng, Tao, Jianlong, Zhou, Zhanfeng, Chai, Lining, Yan, Ying, Liu, Jia, Fu, Rongrong
• Format: Journal Article
• Language: English
• Published: Ireland Elsevier B.V 01.06.2024
• Subjects: Channel attention; Feature correlation matrix; Multi-scale feature fusion; Resting state EEG; Virtual reality motion sickness; Resting state EEG; Virtual reality motion sickness; Channel attention; Feature correlation matrix; Multi-scale feature fusion
• ISSN: 0169-2607; 1872-7565; 1872-7565
• DOI: 10.1016/j.cmpb.2024.108218

•Calculate the inter-lead correlation of the time-domain features extracted by CNN, and convert the time-domain features into spatial features.•The extracted multilayer correlation features are fused to obtain more discriminative multiscale fusion features.•Utilizing the channel attention mechanism to allow the model to focus on features that are more relevant to identifying motion sickness.•The results show that the model is able to provide effective identification of motion sickness and obtains results that exceed those of many classical and advanced models.•The research herein is instructive for the detection of motion sickness and subsequent recovery. Virtual reality motion sickness (VRMS) is a key issue hindering the development of virtual reality technology, and accurate detection of its occurrence is the first prerequisite for solving the issue. In this paper, a convolutional neural network (CNN) EEG detection model based on multi-scale feature correlation is proposed for detecting VRMS. The model uses multi-scale 1D convolutional layers to extract multi-scale temporal features from the multi-lead EEG data, and then calculates the feature correlations of the extracted multi-scale features among all the leads to form the feature adjacent matrixes, which converts the time-domain features to correlation-based brain network features, thus strengthen the feature representation. Finally, the correlation features of each layer are fused. The fused features are then fed into the channel attention module to filter the channels and classify them using a fully connected network. Finally, we recruit subjects to experience 6 different modes of virtual roller coaster scenes, and collect resting EEG data before and after the task to verify the model. Results: The results show that the accuracy, precision, recall and F1-score of this model for the recognition of VRMS are 98.66 %, 98.65 %, 98.68 %, and 98.66 %, respectively. The proposed model outperforms the current classic and advanced EEG recognition models. It shows that this model can be used for the recognition of VRMS based on the resting state EEG.