Tracking the Visual Focus of Attention for a Varying Number of Wandering People

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 30; no. 7; pp. 1212 - 1229
• Main Authors: Smith, K., Ba, S.O., Odobez, J.-M., Gatica-Perez, D.
• Format: Journal Article
• Language: English
• Published: Los Alamitos, CA IEEE 01.07.2008; IEEE Computer Society; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Advertisements; Advertising; Algorithms; Applied sciences; Artificial Intelligence; Attention; Bayesian methods; Cameras; Cognitive science; Computer science; control theory; systems; Computer simulation; Computer systems and distributed systems. User interface; Computer vision; Consumer products; Current measurement; Displays; Exact sciences and technology; Hidden Markov models; Humans; Image Enhancement - methods; Image Interpretation, Computer-Assisted - methods; Image Processing and Computer Vision; Imaging, Three-Dimensional - methods; Inference; Layout; Marketing; Mathematical models; Monte Carlo methods; Movement - physiology; Outdoor; Pattern Recognition, Automated - methods; Pattern recognition. Digital image processing. Computational geometry; Reproducibility of Results; Scene Analysis; Sensitivity and Specificity; Software; Studies; Tracking; Video Recording - methods; Visual; Visual Fields - physiology; Visual Perception - physiology; Scene Analysis; Computer vision; Tracking; Image Processing and Computer Vision; Marketing; Density estimation; Automatic classification; Image processing; consumer products; video analysis; Modeling; Posture; Markov chain; Observer; Bayes network; Cognitive science; Sampling; Localization; Pattern analysis; Monte Carlo method; Head; Motion estimation; Video cameras; Inference; Markov model; State space method; Distributed system; Image analysis; Hidden Markov model; Jumping; Visual attention; Artificial intelligence
• ISSN: 0162-8828; 1939-3539
• DOI: 10.1109/TPAMI.2007.70773

In this paper, we define and address the problem of finding the visual focus of attention for a varying number of wandering people (VFOA-W), determining where a person is looking when their movement is unconstrained. The VFOA-W estimation is a new and important problem with implications in behavior understanding and cognitive science and real-world applications. One such application, presented in this paper, monitors the attention passers-by pay to an outdoor advertisement by using a single video camera. In our approach to the VFOA-W problem, we propose a multiperson tracking solution based on a dynamic Bayesian network that simultaneously infers the number of people in a scene, their body locations, their head locations, and their head pose. For efficient inference in the resulting variable-dimensional state-space, we propose a Reversible-Jump Markov Chain Monte Carlo (RJMCMC) sampling scheme and a novel global observation model, which determines the number of people in the scene and their locations. To determine if a person is looking at the advertisement or not, we propose Gaussian Mixture Model (GMM)-based and Hidden Markov Model (HMM)-based VFOA-W models, which use head pose and location information. Our models are evaluated for tracking performance and ability to recognize people looking at an outdoor advertisement, with results indicating good performance on sequences where up to three mobile observers pass in front of an advertisement.