A radial basis deformable residual convolutional neural model embedded with local multi-modal feature knowledge and its application in cross-subject classification

• Published in: Expert systems with applications Vol. 257; p. 125089
• Main Authors: Li, Jingjing, Zhou, Yanhong, Liu, Tiange, Jung, Tzyy-Ping, Wan, Xianglong, Duan, Dingna, Li, Danyang, Yu, Hao, Song, Haiqing, Dong, Xianling, Wen, Dong
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 10.12.2024
• Subjects: Brain regions correlation; Cross-subject; Embedded feature knowledge; Radial basis deformable residual convolutional neural networks; Brain regions correlation; Radial basis deformable residual convolutional neural networks; Embedded feature knowledge; Cross-subject
• DOI: 10.1016/j.eswa.2024.125089
• ISSN: 0957-4174

In the spatial cognition and emotion recognition tasks based on electroencephalography (EEG), the signal information representation of the single modal is incomplete because of the significant inter-subject differences in the EEG signals, resulting in low generalization performance of classification models. To address this issue, we propose a radial basis deformable residual convolutional neural networks model embedded with local multi-modal feature knowledge (RBDRCNN-LMFK). The RBDRCNN leverages Euclidean distance alignment, deformable convolution, depth-separable convolution, and residual connection modules for effective EEG feature extraction. The embedded local feature knowledge algorithm enables the effective combination of multi-modal information. To further validate the effectiveness of the algorithm, we used the left-one-subject-out cross-validation algorithm on the virtual city walking (VCW), SJTU Emotion EEG Dataset IV (SEED IV), and Building block gesture recognition (BBGR) datasets for spatial cognition and emotion recognition tasks. The average accuracy of the VCW was 97.84 % using the kNN classifier, surpassing the Concate fusion method’s 96.62 %. The average accuracy for the SEED IV dataset was 87.56 %, higher than the Concate Fusion’s 69.97 %. On the BBGR dataset, the kNN classifier achieved an average accuracy of 87.80 %, compared to 85.31 % with the Concate fusion. The results show that the model enhances the recognition of biologically significant features in EEG signals by embedding local features and increases the correlation between different brain regions associated with the task. This work illustrates a promising direction of using deep learning models to discover effective task-related features from highly diverse EEG signals and enhance brain regions’ correlation through multi-modal knowledge embedding.