Saliency Detection in Face Videos: A Data-Driven Approach

• Published in: IEEE transactions on multimedia Vol. 20; no. 6; pp. 1335 - 1349
• Main Authors: Xu, Mai, Ren, Yun, Wang, Zulin, Liu, Jingxian, Tao, Xiaoming
• Format: Journal Article
• Language: English
• Published: IEEE 01.06.2018
• Subjects: Face; Face video; Feature extraction; Gaussian mixture model; Gaze tracking; Mouth; Object detection; Videos; visual attention; Visualization
• ISSN: 1520-9210; 1941-0077
• DOI: 10.1109/TMM.2017.2767784

Recently, videoconferencing has been popular in multimedia systems, such as FaceTime and Skype. In videoconferencing, almost every frame contains a human face. Therefore, it is important to predict human visual attention on face videos by saliency detection, as saliency may be used as a guide to the region of interest for the content-based applications of face videos. In this paper, we propose a data-driven approach for saliency detection in face videos. From the data-driven perspective, we first establish an eye-tracking database that contains fixations of 76 face videos viewed by 40 subjects. Upon the analysis of our database, we find that visual attention is significantly attracted by faces in videos. More important, the attention distribution within face regions varies with regard to mouth movement. Since previous works have investigated that it is efficient to model face saliency in still images using a Gaussian mixture model (GMM), the variation of visual attention in videos can be modeled by dynamic GMM (DGMM). Accordingly, we propose adopting the particle filter (PF) in modeling DGMM for saliency detection of face videos, which is called PF-DGMM. Finally, the experimental results show that our PF-DGMM approach significantly outperforms other state-of-the-art approaches in saliency detection of face videos.