Direct Optical-Flow-Aware Computational Framework for 3D Reconstruction

• Published in: IEEE access Vol. 7; pp. 169518 - 169527
• Main Authors: Hu, Huijuan, Chen, Pei
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptive optics; Cameras; Datasets; Estimation; fast guided interpolation; Image segmentation; Interpolation; Linear programming; Optical flow; Optical flow (image analysis); Optical imaging; Optimization; Pedestrians; Reconstruction; structure from motion; the left-right consistency constraint; the soft segment constraint; Three-dimensional displays
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2019.2954917

In this paper, a direct computational method is presented which combines optical flow and structure from motion (SfM) by putting the SfM problem in the framework of optical flow estimation. In other word, the optical flow is reparametrized in term of the camera's motion and scene's depth, resulting in a similar variation optimization as in optical flow estimation. Meanwhile, three techniques are proposed to improve the accuracy and robustness of the direct approach, including the fast guided interpolation (FGI), the left-right consistency constraint and the soft segment constraint. Experimental results on the Middlebury dataset and KITTI2012 dataset show that the proposed approach can achieve highly-accurate 3D reconstruction with the dense and smooth surface which results in a state-of-the-art performance in optical flow.