ADABase: A Multimodal Dataset for Cognitive Load Estimation

• Published in: Sensors (Basel, Switzerland) Vol. 23; no. 1; p. 340
• Main Authors: Oppelt, Maximilian P., Foltyn, Andreas, Deuschel, Jessica, Lang, Nadine R., Holzer, Nina, Eskofier, Bjoern M., Yang, Seung Hee
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI AG 28.12.2022; MDPI
• Subjects: affective computing; Artificial intelligence; Automation; Automobile Driving - psychology; autonomous driving; Business metrics; Cognition; Cognitive load; Datasets; Emotions; Humans; Machine Learning; multimodal dataset; Physiology; Reaction Time; Vehicles; Workload; Workloads; cognitive load; machine learning; affective computing; multimodal dataset; autonomous driving
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s23010340

Driver monitoring systems play an important role in lower to mid-level autonomous vehicles. Our work focuses on the detection of cognitive load as a component of driver-state estimation to improve traffic safety. By inducing single and dual-task workloads of increasing intensity on 51 subjects, while continuously measuring signals from multiple modalities, based on physiological measurements such as ECG, EDA, EMG, PPG, respiration rate, skin temperature and eye tracker data, as well as behavioral measurements such as action units extracted from facial videos, performance metrics like reaction time and subjective feedback using questionnaires, we create ADABase (Autonomous Driving Cognitive Load Assessment Database) As a reference method to induce cognitive load onto subjects, we use the well-established n-back test, in addition to our novel simulator-based k-drive test, motivated by real-world semi-autonomously vehicles. We extract expert features of all measurements and find significant changes in multiple modalities. Ultimately we train and evaluate machine learning algorithms using single and multimodal inputs to distinguish cognitive load levels. We carefully evaluate model behavior and study feature importance. In summary, we introduce a novel cognitive load test, create a cognitive load database, validate changes using statistical tests, introduce novel classification and regression tasks for machine learning and train and evaluate machine learning models.