EEG-Based Affect and Workload Recognition in a Virtual Driving Environment for ASD Intervention

• Published in: IEEE transactions on biomedical engineering Vol. 65; no. 1; pp. 43 - 51
• Main Authors: Fan, Jing, Wade, Joshua W., Key, Alexandra P., Warren, Zachary E., Sarkar, Nilanjan
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.01.2018; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adolescent; Adolescents; Affective computing; Algorithms; Autism; Autism Spectrum Disorder - physiopathology; Autism Spectrum Disorder - psychology; Autism Spectrum Disorder - rehabilitation; Automobile Driving; Bins; Boredom; Brain; Brain models; Classification; Computer applications; Computer simulation; Data acquisition; Driving ability; EEG; electroencephalogram; Electroencephalography; Electroencephalography - methods; Emotional behavior; Feature extraction; Female; Frequencies; Frustration; Human-computer interface; Humans; Implants; Male; mental workload recognition; Signal Processing, Computer-Assisted; Solid modeling; Training; Variable speed drives; Virtual environments; Virtual reality; virtual reality-based driving simulator; Workload; Workloads
• ISSN: 0018-9294; 1558-2531; 1558-2531
• DOI: 10.1109/TBME.2017.2693157

Objective: To build group-level classification models capable of recognizing affective states and mental workload of individuals with autism spectrum disorder (ASD) during driving skill training. Methods: Twenty adolescents with ASD participated in a six-session virtual reality driving simulator-based experiment, during which their electroencephalogram (EEG) data were recorded alongside driving events and a therapist's rating of their affective states and mental workload. Five feature generation approaches including statistical features, fractal dimension features, higher order crossings (HOC)-based features, power features from frequency bands, and power features from bins (Δf = 2 Hz) were applied to extract relevant features. Individual differences were removed with a two-step feature calibration method. Finally, binary classification results based on the k-nearest neighbors algorithm and univariate feature selection method were evaluated by leave-one-subject-out nested cross-validation to compare feature types and identify discriminative features. Results: The best classification results were achieved using power features from bins for engagement (0.95) and boredom (0.78), and HOC-based features for enjoyment (0.90), frustration (0.88), and workload (0.86). Conclusion: Offline EEG-based group-level classification models are feasible for recognizing binary low and high intensity of affect and workload of individuals with ASD in the context of driving. However, while promising the applicability of the models in an online adaptive driving task requires further development. Significance: The developed models provide a basis for an EEG-based passive brain computer interface system that has the potential to benefit individuals with ASD with an affect and workload-based individualized driving skill training intervention.