3D reconstruction of spherical images based on incremental structure from motion

• Published in: International journal of remote sensing Vol. 45; no. 8; pp. 2596 - 2621
• Main Authors: Jiang, San, You, Kan, Chen, Wu, Weng, Duojie, Li, Yaxin
• Format: Journal Article
• Language: English
• Published: London Taylor & Francis 17.04.2024; Taylor & Francis Ltd
• Subjects: 3D reconstruction; Algorithms; Bundle adjustment; Cameras; Canyons; Cost function; Data collection; equirectangular projection; image matching; Image reconstruction; Landforms; Low altitude; Motion perception; Open source software; Photogrammetry; Remote sensing; Software packages; Source code; Spherical image; Street canyons; structure from motion; Three dimensional motion; Unmanned aerial vehicles; Workflow
• ISSN: 0143-1161; 1366-5901
• DOI: 10.1080/01431161.2024.2334810

3D reconstruction plays an increasingly important role in modern photogrammetric systems. Conventional satellite or aerial-based remote sensing (RS) platforms can provide the necessary data sources for the 3D reconstruction of large-scale landforms and cities. Even with low-altitude UAVs (Unmanned Aerial Vehicles), 3D reconstruction in complicated situations, such as urban canyons and indoor scenes, is challenging due to the frequent tracking failures of camera frames and high data collection costs. Recently, spherical images have been extensively exploited due to the capability of recording surrounding environments using one camera exposure. Classical 3D reconstruction pipelines, however, cannot be used for spherical images. Besides, there exist few software packages for the 3D reconstruction of spherical images. Based on the imaging geometry of spherical cameras, this study investigates the algorithms for relative orientation using spherical correspondences, absolute orientation using 3D correspondences between the scene and spherical points, as well as cost functions for BA (bundle adjustment) optimization. Besides, an incremental SfM (Structure from Motion) workflow has been proposed for spherical images by using the above-mentioned algorithms. The proposed solution is finally verified by using three spherical datasets captured by both consumer-grade and professional spherical cameras. The results demonstrate that the proposed SfM workflow can achieve the successful 3D reconstruction of complex scenes and provide useful clues for the implementation in open-source software packages. The source code would be publicly available.