The ZuCo benchmark on cross-subject reading task classification with EEG and eye-tracking data

• Published in: Frontiers in psychology Vol. 13; p. 1028824
• Main Authors: Hollenstein, Nora, Tröndle, Marius, Plomecka, Martyna, Kiegeland, Samuel, Özyurt, Yilmazcan, Jäger, Lena A., Langer, Nicolas
• Format: Journal Article
• Language: English
• Published: Switzerland Frontiers Media S.A 12.01.2023
• Subjects: cross-subject evaluation; EEG; eye-tracking; machine learning; Psychology; reading research; reading task classification; reading research; reading task classification; eye-tracking; machine learning; EEG; cross-subject evaluation
• ISSN: 1664-1078; 1664-1078
• DOI: 10.3389/fpsyg.2022.1028824

We present a new machine learning benchmark for reading task classification with the goal of advancing EEG and eye-tracking research at the intersection between computational language processing and cognitive neuroscience. The benchmark task consists of a cross-subject classification to distinguish between two reading paradigms: normal reading and task-specific reading. The data for the benchmark is based on the Zurich Cognitive Language Processing Corpus (ZuCo 2.0), which provides simultaneous eye-tracking and EEG signals from natural reading of English sentences. The training dataset is publicly available, and we present a newly recorded hidden testset. We provide multiple solid baseline methods for this task and discuss future improvements. We release our code and provide an easy-to-use interface to evaluate new approaches with an accompanying public leaderboard: www.zuco-benchmark.com .