Hierarchical Improvement of Foreground Segmentation Masks in Background Subtraction

• Published in: IEEE transactions on circuits and systems for video technology Vol. 29; no. 6; pp. 1645 - 1658
• Main Authors: Ortego, Diego, Sanmiguel, Juan C., Martinez, Jose M.
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.06.2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptation models; Algorithms; background subtraction; Computer vision; Conditional random fields; Fitness; foreground quality; Foreground segmentation improvement; Image quality; Image segmentation; Lighting; Masks; Merging; Motion segmentation; Optical imaging; Pixels; post-processing; Post-production processing; Subtraction; Videos
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2018.2851440

A plethora of algorithms have been defined for foreground segmentation, a fundamental stage for many computer vision applications. In this paper, we propose a post-processing framework to improve the foreground segmentation performance of background subtraction algorithms. We define a hierarchical framework for extending segmented foreground pixels to undetected foreground object areas and for removing erroneously segmented foreground. First, we create a motion-aware hierarchical image segmentation of each frame that prevents merging foreground and background image regions. Then, we estimate the quality of the foreground mask through the fitness of the binary regions in the mask and the hierarchy of segmented regions. Finally, the improved foreground mask is obtained as an optimal labeling by jointly exploiting foreground quality and spatial color relations in a pixel-wise fully connected conditional random field. Experiments are conducted over four large and heterogeneous data sets with varied challenges (CDNET2014, LASIESTA, SABS, and BMC) demonstrating the capability of the proposed framework to improve background subtraction results.