Saliency-Aware Video Object Segmentation

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 40; no. 1; pp. 20 - 33
• Main Authors: Wenguan Wang, Jianbing Shen, Ruigang Yang, Porikli, Fatih
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.01.2018; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Energy conservation; geodesic distance; Image segmentation; Motion segmentation; Object segmentation; Pixels; Proposals; Salience; Segmentation; Skeleton; Smoothness; spatiotemporal object prior; Spatiotemporal phenomena; State of the art; Trajectory; video object segmentation; Video saliency
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2017.2662005

Video saliency, aiming for estimation of a single dominant object in a sequence, offers strong object-level cues for unsupervised video object segmentation. In this paper, we present a geodesic distance based technique that provides reliable and temporally consistent saliency measurement of superpixels as a prior for pixel-wise labeling. Using undirected intra-frame and inter-frame graphs constructed from spatiotemporal edges or appearance and motion, and a skeleton abstraction step to further enhance saliency estimates, our method formulates the pixel-wise segmentation task as an energy minimization problem on a function that consists of unary terms of global foreground and background models, dynamic location models, and pairwise terms of label smoothness potentials. We perform extensive quantitative and qualitative experiments on benchmark datasets. Our method achieves superior performance in comparison to the current state-of-the-art in terms of accuracy and speed.