Uncertainty-aware Gaze Tracking for Assisted Living Environments

• Published in: IEEE transactions on image processing Vol. 32; p. 1
• Main Authors: Her, Paris, Manderle, Logan, Dias, Philipe A., Medeiros, Henry, Odone, Francesca
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.01.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Datasets; Eye-Tracking Technology; Fixation, Ocular; Humans; Kalman filters; Neural networks; Neural Networks, Computer; Tracking; Uncertainty
• ISSN: 1057-7149; 1941-0042; 1941-0042
• DOI: 10.1109/TIP.2023.3253253

Effective assisted living environments must be able to infer how their occupants interact in a variety of scenarios. Gaze direction provides strong indications of how a person engages with the environment and its occupants. In this paper, we investigate the problem of gaze tracking in multi-camera assisted living environments. We propose a gaze tracking method based on predictions generated by a neural network regressor that relies only on the relative positions of facial keypoints to estimate gaze. For each gaze prediction, our regressor also provides an estimate of its own uncertainty, which is used to weigh the contribution of previously estimated gazes within a tracking framework based on an angular Kalman filter. Our gaze estimation neural network uses confidence gated units to alleviate keypoint prediction uncertainties in scenarios involving partial occlusions or unfavorable views of the subjects. We evaluate our method using videos from the MoDiPro dataset, which we acquired in a real assisted living facility, and on the publicly available GazeFollow and Gaze360 datasets. Experimental results show that our gaze estimation network outperforms sophisticated state-of-the-art methods, while additionally providing uncertainty predictions that are highly correlated with the actual angular error of the corresponding estimates. Finally, an analysis of the temporal integration performance of our method demonstrates that it generates accurate and temporally stable gaze predictions.