COLET: A dataset for COgnitive workLoad estimation based on eye-tracking

• Published in: Computer methods and programs in biomedicine Vol. 224; p. 106989
• Main Authors: Ktistakis, Emmanouil, Skaramagkas, Vasileios, Manousos, Dimitris, Tachos, Nikolaos S., Tripoliti, Evanthia, Fotiadis, Dimitrios I., Tsiknakis, Manolis
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.09.2022
• Subjects: Affective computing; Cognitive workload; Eye movements; Eye-tracking; Machine learning; Workload classification; Affective computing; Eye movements; Cognitive workload; Workload classification; Eye-tracking; Machine learning
• ISSN: 0169-2607; 1872-7565
• DOI: 10.1016/j.cmpb.2022.106989

•The dataset combines both objective and subjective performance evaluation.•Wide variety of eye tracking data, high number of participants.•Statistically significant correlations of eye features with cognitive workload tasks.•Encouraging results regarding cognitive workload level classification based on eye tracking data collected.•Publicly available to the academic community. Background and Objective: The cognitive workload is an important component in performance psychology, ergonomics, and human factors. Publicly available datasets are scarce, making it difficult to establish new approaches and comparative studies. In this work, COLET-COgnitive workLoad estimation based on Eye-Tracking dataset is presented. Methods: Forty-seven (47) individuals’ eye movements were monitored as they solved puzzles involving visual search activities of varying complexity and duration. The participants’ cognitive workload level was evaluated with the subjective test of NASA-TLX and this score is used as an annotation of the activity. Extensive data analysis was performed in order to derive eye and gaze features from low-level eye recorded metrics, and a range of machine learning models were evaluated and tested regarding the estimation of the cognitive workload level. Results: The activities induced four different levels of cognitive workload. Multi tasking and time pressure have induced a higher level of cognitive workload than the one induced by single tasking and absence of time pressure. Multi tasking had a significant effect on 17 eye features while time pressure had a significant effect on 7 eye features. Both binary and multi-class identification attempts were performed by testing a variety of well-known classifiers, resulting in encouraging results towards cognitive workload levels estimation, with up to 88% correct predictions between low and high cognitive workload. Conclusions: Machine learning analysis demonstrated potential in discriminating cognitive workload levels using only eye-tracking characteristics. The proposed dataset includes a much higher sample size and a wider spectrum of eye and gaze metrics than other similar datasets, allowing for the examination of their relations with various cognitive states.