I-MPN: Inductive Message Passing Network for Efficient Human-in-the-Loop Annotation of Mobile Eye Tracking Data

Comprehending how humans process visual information in dynamic settings is crucial for psychology and designing user-centered interactions. While mobile eye-tracking systems combining egocentric video and gaze signals can offer valuable insights, manual analysis of these recordings is time-intensive...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Le, Hoang H, Nguyen, Duy M H, Omair Shahzad Bhatti, Kopacsi, Laszlo, Ngo, Thinh P, Nguyen, Binh T, Barz, Michael, Sonntag, Daniel
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 07.07.2024
Subjects
Algorithms
Annotations
Detectors
Eye movements
Machine learning
Message passing
Object recognition
Tracking systems
Online AccessGet full text

Cover

More Information
Summary:Comprehending how humans process visual information in dynamic settings is crucial for psychology and designing user-centered interactions. While mobile eye-tracking systems combining egocentric video and gaze signals can offer valuable insights, manual analysis of these recordings is time-intensive. In this work, we present a novel human-centered learning algorithm designed for automated object recognition within mobile eye-tracking settings. Our approach seamlessly integrates an object detector with a spatial relation-aware inductive message-passing network (I-MPN), harnessing node profile information and capturing object correlations. Such mechanisms enable us to learn embedding functions capable of generalizing to new object angle views, facilitating rapid adaptation and efficient reasoning in dynamic contexts as users navigate their environment. Through experiments conducted on three distinct video sequences, our interactive-based method showcases significant performance improvements over fixed training/testing algorithms, even when trained on considerably smaller annotated samples collected through user feedback. Furthermore, we demonstrate exceptional efficiency in data annotation processes and surpass prior interactive methods that use complete object detectors, combine detectors with convolutional networks, or employ interactive video segmentation.
ISSN:2331-8422