MoGaze: A Dataset of Full-Body Motions that Includes Workspace Geometry and Eye-Gaze

• Published in: IEEE robotics and automation letters Vol. 6; no. 2; pp. 367 - 373
• Main Authors: Kratzer, Philipp, Bihlmaier, Simon, Midlagajni, Niteesh Balachandra, Prakash, Rohit, Toussaint, Marc, Mainprice, Jim
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.04.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Cameras; Datasets; Datasets for human motion; Eye movements; Human motion; human-centered robotics; Linux; modeling and simulating humans; Motion capture; Pupils; Robot dynamics; Robots; Task analysis; Three dimensional models; Three-dimensional displays; Tracking; Tracking devices
• ISSN: 2377-3766; 2377-3766
• DOI: 10.1109/LRA.2020.3043167

As robots become more present in open human environments, it will become crucial for robotic systems to understand and predict human motion. Such capabilities depend heavily on the quality and availability of motion capture data. However, existing datasets of full-body motion rarely include 1) long sequences of manipulation tasks, 2) the 3D model of the workspace geometry, and 3) eye-gaze, which are all important when a robot needs to predict the movements of humans in close proximity. Hence, in this letter, we present a novel dataset of full-body motion for everyday manipulation tasks, which includes the above. The motion data was captured using a traditional motion capture system based on reflective markers. We additionally captured eye-gaze using a wearable pupil-tracking device. As we show in experiments, the dataset can be used for the design and evaluation of full-body motion prediction algorithms. Furthermore, our experiments show eye-gaze as a powerful predictor of human intent. The dataset includes 180 min of motion capture data with 1627 pick and place actions being performed. It is available at https://humans-to-robots-motion.github.io/mogaze/ MoGaze, Dataset and is planned to be extended to collaborative tasks with two humans in the near future.