Fusion of Spatial, Temporal, and Spectral EEG Signatures Improves Multilevel Cognitive Load Prediction

• Published in: IEEE transactions on human-machine systems Vol. 53; no. 2; pp. 357 - 366
• Main Authors: Liu, Yingxin, Yu, Yang, Ye, Zeqi, Li, Ming, Zhang, Yifan, Zhou, Zongtan, Hu, Dewen, Zeng, Ling-Li
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.04.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Classification; Cognitive load; Cognitive tasks; Electroencephalography; electroencephalography (EEG); Feature extraction; functional connectivity; Games; Indexes; Man-machine systems; Memory tasks; microstate; Multilevel; power spectral density (PSD); Spectral signatures; Task analysis; Telematics; Topology
• DOI: 10.1109/THMS.2023.3235003
• ISSN: 2168-2291

Cognitive load prediction is one of the most important issues in the nascent field of neuroergonomics, and it has significant value in real-world applications. Most of the previous studies of cognitive load prediction only utilized electroencephalography (EEG)-based spectral signatures or interchannel connectivity, ignoring abundant temporal microstate features, which may represent the transient topologies of EEG signals. Furthermore, previous studies have mostly focused on the binary-level classification of cognitive load for single-type cognitive tasks. To date, there are few studies on the multilevel prediction of cognitive load during mixed cognitive tasks. Here, we first designed a new paradigm termed the "finding fault game," mixing multiple tasks of memory, counting, and visual search, and then developed a multidimensional analysis framework to improve cognitive load prediction using a fusion of spatial, temporal, and spectral EEG features. Specifically, EEG-based functional connectivity, microstates and power spectral densities (PSD) were calculated for three cognitive load levels. Twelve adult subjects participated in the study. The experimental results show that increased cognitive load was associated with elevated theta and degraded alpha power and significant changes in interchannel connectivity and microstates, and that fusing the three types of EEG features improved the performance of three-level cognitive load prediction, achieving the accuracies of greater than 80% in the cross-validation, real-time, and over-time prediction. The findings suggest that all three types of EEG features can serve as signatures of cognitive load and that their fusion can improve multilevel prediction.