GPU-friendly multi-view stereo reconstruction using surfel representation and graph cuts

• Published in: Computer vision and image understanding Vol. 115; no. 5; pp. 620 - 634
• Main Authors: Chang, Ju Yong, Park, Haesol, Park, In Kyu, Lee, Kyoung Mu, Lee, Sang Uk
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 01.05.2011
• Subjects: 3D shape reconstruction; Algorithms; Construction; GPU; Graph cuts; Graphs; Massive parallel processing; Multi-view stereo; Optimization; Orientation; Representations; Surfel; Three dimensional; Massive parallel processing; Multi-view stereo; Surfel; 3D shape reconstruction; GPU; Graph cuts
• ISSN: 1077-3142; 1090-235X
• DOI: 10.1016/j.cviu.2010.11.017

► We effectively combine a surfel-based surface representation with the optimization framework using graph cuts. ► The surface details are more accurately reconstructed by using photo-consistently refined surfels rather than regularly shaped voxels for the graph cuts optimization. ► Parallel processing of the proposed method is designed and implemented completely on GPU using the latest CUDA technology, which runs more than 100 times faster than on comparable CPU. In this paper, we present a new surfel (surface element) based multi-view stereo algorithm that runs entirely on GPU. We utilize the flexibility of surfel-based 3D shape representation and global optimization by graph cuts in the same framework. Unlike previous works, the algorithm is optimized to massive parallel processing on GPU. First, we construct surfel candidates by local stereo matching and voting. After refining the position and orientation of the surfel candidates, we extract the optimal surfels by employing graph cuts under photo-consistency and surfel orientation constraints. In contrast to the conventional voxel based methods, the proposed algorithm utilizes more accurate photo-consistency and reconstructs the 3D shape up to sub-voxel accuracy. The orientation of the constructed surfel candidates imposes an effective constraint that reduces the effect of the minimal surface bias. The entire processing pipeline is implemented on the latest GPU to significantly speed up the processing. The experimental results show that the proposed approach reconstructs the 3D shape of an object accurately and efficiently, which runs more than 100 times faster than on CPU.