High-fidelity eye, head, body, and world tracking with a wearable device

• Published in: Behavior research methods Vol. 56; no. 1; pp. 32 - 42
• Main Authors: DuTell, Vasha, Gibaldi, Agostino, Focarelli, Giulia, Olshausen, Bruno A., Banks, Martin S.
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.01.2024; Springer Nature B.V
• Subjects: Behavioral Science and Psychology; Binocular vision; Cameras; Cognitive Psychology; Data Collection; Eye; Humans; Psychology; Sensors; Vision, Ocular; Visual stimuli; Wearable Electronic Devices; Human vision; Hardware; Natural scene statistics; Eye tracking
• ISSN: 1554-3528; 1554-351X; 1554-3528
• DOI: 10.3758/s13428-022-01888-3

We describe the design and performance of a high-fidelity wearable head-, body-, and eye-tracking system that offers significant improvement over previous such devices. This device’s sensors include a binocular eye tracker, an RGB-D scene camera, a high-frame-rate scene camera, and two visual odometry sensors, for a total of ten cameras, which we synchronize and record from with a data rate of over 700 MB/s. The sensors are operated by a mini-PC optimized for fast data collection, and powered by a small battery pack. The device records a subject’s eye, head, and body positions, simultaneously with RGB and depth data from the subject’s visual environment, measured with high spatial and temporal resolution. The headset weighs only 1.4 kg, and the backpack with batteries 3.9 kg. The device can be comfortably worn by the subject, allowing a high degree of mobility. Together, this system overcomes many limitations of previous such systems, allowing high-fidelity characterization of the dynamics of natural vision.