Continuous 3D Label Stereo Matching Using Local Expansion Moves

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 40; no. 11; pp. 2725 - 2739
• Main Authors: Taniai, Tatsunori, Matsushita, Yasuyuki, Sato, Yoichi, Naemura, Takeshi
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.11.2018; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: 3D reconstruction; Acceleration; Curvature; discrete-continuous optimization; Filtration; graph cuts; Image segmentation; Labels; Localization; Markov processes; Markov random fields; Matching; Optimization; Parallel processing; Pattern matching; Position (location); Propagation; Proposals; Regularization; Stereo vision; Three-dimensional displays
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2017.2766072

We present an accurate stereo matching method using local expansion moves based on graph cuts. This new move-making scheme is used to efficiently infer per-pixel 3D plane labels on a pairwise Markov random field (MRF) that effectively combines recently proposed slanted patch matching and curvature regularization terms. The local expansion moves are presented as many <inline-formula><tex-math notation="LaTeX">\alpha</tex-math> <inline-graphic xlink:href="taniai-ieq1-2766072.gif"/> </inline-formula>-expansions defined for small grid regions. The local expansion moves extend traditional expansion moves by two ways: localization and spatial propagation. By localization, we use different candidate <inline-formula><tex-math notation="LaTeX">\alpha</tex-math> <inline-graphic xlink:href="taniai-ieq2-2766072.gif"/> </inline-formula>-labels according to the locations of local <inline-formula><tex-math notation="LaTeX">\alpha</tex-math> <inline-graphic xlink:href="taniai-ieq3-2766072.gif"/> </inline-formula>-expansions. By spatial propagation, we design our local <inline-formula><tex-math notation="LaTeX">\alpha</tex-math> <inline-graphic xlink:href="taniai-ieq4-2766072.gif"/> </inline-formula>-expansions to propagate currently assigned labels for nearby regions. With this localization and spatial propagation, our method can efficiently infer MRF models with a continuous label space using randomized search. Our method has several advantages over previous approaches that are based on fusion moves or belief propagation; it produces submodular moves deriving a subproblem optimality ; it helps find good, smooth, piecewise linear disparity maps; it is suitable for parallelization; it can use cost-volume filtering techniques for accelerating the matching cost computations. Even using a simple pairwise MRF, our method is shown to have best performance in the Middlebury stereo benchmark V2 and V3.