Object-Based Multiple Foreground Segmentation in RGBD Video

• Published in: IEEE transactions on image processing Vol. 26; no. 3; pp. 1418 - 1427
• Main Authors: Huazhu Fu, Dong Xu, Lin, Stephen
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.03.2017; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Constraint modelling; Electronic mail; Graph theory; Image color analysis; Image segmentation; Labeling; Linear programming; multiple object segmentation; Object recognition; Proposals; RGBD segmentation; RGBD video saliency; Segmentation; Video segmentation
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2017.2651369

We present an RGB and Depth (RGBD) video segmentation method that takes advantage of depth data and can extract multiple foregrounds in the scene. This video segmentation is addressed as an object proposal selection problem formulated in a fully-connected graph, where a flexible number of foregrounds may be chosen. In our graph, each node represents a proposal, and the edges model intra-frame and inter-frame constraints on the solution. The proposals are selected based on an RGBD video saliency map in which depth-based features are utilized to enhance the identification of foregrounds. Experiments show that the proposed multiple foreground segmentation method outperforms related techniques, and the depth cue serves as a helpful complement to RGB features. Moreover, our method provides performance comparable to the state-of-the-art RGB video segmentation techniques on regular RGB videos with estimated depth maps.