CLARE: Cognitive Load Assessment in Real-time with Multimodal Data

• Published in: IEEE transactions on cognitive and developmental systems pp. 1 - 13
• Main Authors: Bhatti, Anubhav, Angkan, Prithila, Behinaein, Behnam, Mahmud, Zunayed, Rodenburg, Dirk, Braund, Heather, Mclellan, P. James, Ruberto, Aaron, Harrison, Geoffery, Wilson, Daryl, Szulewski, Adam, Howes, Dan, Etemad, Ali, Hungler, Paul
• Format: Journal Article
• Language: English
• Published: IEEE 2025
• Subjects: Affective Computing; Biomedical monitoring; Cognitive load; Deep learning; ECG; EDA; EEG; Electrocardiography; Electroencephalography; Electromyography; GAZE; Human factors; Multimodal Dataset; Physiology; Real-time systems; Wearable devices
• ISSN: 2379-8920; 2379-8939
• DOI: 10.1109/TCDS.2025.3555517

We present a novel multimodal dataset for Cognitive Load Assessment in REal-time (CLARE). The dataset contains physiological and Gaze data from 24 participants with self-reported cognitive load scores as ground-truth labels. The dataset includes four modalities: Electrocardiography (ECG), Electrodermal Activity (EDA), Electroencephalogram (EEG), and Gaze tracking. Each participant completed four nine-minute sessions using the MATB-II software, a computer-based mental workload task. The sessions were divided into one-minute segments of varying complexity to induce different levels of cognitive load. During the experiment, participants reported their cognitive load every 10 seconds. For the dataset, we also provide benchmark binary classification results with machine learning and deep learning models on two different evaluation schemes, namely, 10-fold and leave-one-subject-out (LOSO) cross-validation. Benchmark results show that for 10-fold evaluation, the Transformer based deep learning model achieves the best classification performance with ECG, EDA, and Gaze. In contrast, for LOSO, the best performance is achieved by the deep learning model with ECG, EDA, and EEG.