Goal-oriented top-down probabilistic visual attention model for recognition of manipulated objects in egocentric videos

• Published in: Signal processing. Image communication Vol. 39; no. Part B; pp. 418 - 431
• Main Authors: Buso, Vincent, González-Díaz, Iván, Benois-Pineau, Jenny
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.11.2015; Elsevier
• Series: Signal Processing: Image Communication
• Subjects: Assessments; Computer Science; Egocentric vision; Engineering Sciences; Image processing; Mathematical models; Multimedia; Object recognition; Probabilistic methods; Probability theory; Saliency maps; State of the art; Video processing; Vision modelling; Visual; Saliency maps; Egocentric vision; Video processing; Image processing; Object recognition; Vision modelling
• ISSN: 0923-5965; 1879-2677
• DOI: 10.1016/j.image.2015.05.006

We propose a new top down probabilistic saliency model for egocentric video content. It aims to predict top-down visual attention maps focused on manipulated objects, that are then used for psycho-visual weighting of features in the problem of manipulated object recognition. The model is probabilistically defined using both global and local appearance features extracted from automatically segmented arm areas and objects. A psycho-visual experiment has been conducted in a guided framework that compares our proposal and other popular state-of-the-art models with respect to human gaze fixations. The obtained results show that our approach outperforms several popular bottom-up saliency approaches in a well-known egocentric dataset. Furthermore, an additional task-driven assessment for object recognition in egocentric video reveals that the proposed method improves the performance of several state-of-the-art techniques for object detection. •We propose a new top down probabilistic saliency model for egocentric video content.•Top-down saliency maps focused on manipulated objects based on the arms positions.•Psycho-visual experiment conducted demonstrating our proposal increased performances over SOA saliency models.•SOA performances in manipulated object recognition for this dataset.