A Fully-Connected Layered Model of Foreground and Background Flow

• Published in: 2013 IEEE Conference on Computer Vision and Pattern Recognition pp. 2451 - 2458
• Main Authors: Deqing Sun, Wulff, Jonas, Sudderth, Erik B., Pfister, Hanspeter, Black, Michael J.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.06.2013
• Subjects: Approximation algorithms; Approximation methods; Computational modeling; Estimation; Image segmentation; Motion segmentation; Optical imaging
• ISSN: 1063-6919; 1063-6919
• DOI: 10.1109/CVPR.2013.317

Layered models allow scene segmentation and motion estimation to be formulated together and to inform one another. Traditional layered motion methods, however, employ fairly weak models of scene structure, relying on locally connected Ising/Potts models which have limited ability to capture long-range correlations in natural scenes. To address this, we formulate a fully-connected layered model that enables global reasoning about the complicated segmentations of real objects. Optimization with fully-connected graphical models is challenging, and our inference algorithm leverages recent work on efficient mean field updates for fully-connected conditional random fields. These methods can be implemented efficiently using high-dimensional Gaussian filtering. We combine these ideas with a layered flow model, and find that the long-range connections greatly improve segmentation into figure-ground layers when compared with locally connected MRF models. Experiments on several benchmark datasets show that the method can recover fine structures and large occlusion regions, with good flow accuracy and much lower computational cost than previous locally-connected layered models.