Cross-modal cognitive load evaluation method based on dual residual network

• Main Authors: OUYANG GAOXIANG, LI XIAOLI, GU YUE, YAN YILEI, ZHANG XINPENG
• Format: Patent
• Language: Chinese; English
• Published: 08.09.2023
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; ELECTRIC DIGITAL DATA PROCESSING; PHYSICS

The invention provides a cross-modal cognitive load assessment method based on a dual residual network, which effectively improves the accuracy of cognitive load assessment, and comprises the following steps: firstly, acquiring EEG (electroencephalogram) data and fNIRS (functional near infrared spectroscopy) data, and then respectively carrying out preprocessing operations such as band-pass filtering and down-sampling on the two different data; the method comprises the following steps: preprocessing data, taking the preprocessed data as input, training by utilizing a spatial-temporal feature learning module to obtain feature information, fusing two kinds of feature information to obtain high-dimensional spatial-temporal features, extracting multi-modal features by constructing a dual residual network, and finally realizing cognitive load classification by adopting a full connection layer. According to the method, manual feature extraction can be replaced, cross-modal information features are fused by building