Challenges in Wide-area Structure-from-motion

• Published in: IPSJ Transactions on Computer Vision and Applications Vol. 2; pp. 105 - 120
• Main Authors: Pollefeys, Marc, Frahm, Jan-Michael, Fraundorfer, Friedrich, Zach, Christopher, Wu, Changchang, Clipp, Brian, Gallup, David
• Format: Journal Article
• Language: English
• Published: Information Processing Society of Japan 2010
• ISSN: 1882-6695; 1882-6695
• DOI: 10.2197/ipsjtcva.2.105

The topic of this paper is wide area structure from motion. We first describe recent progress in obtaining large-scale 3D visual models from images. Our approach consists of a multi-stage processing pipeline, which can process a recorded video stream in real-time on standard PC hardware by leveraging the computational power of the graphics processor. The output of this pipeline is a detailed textured 3D model of the recorded area. The approach is demonstrated on video data recorded in Chapel Hill containing more than a million frames. While for these results GPS and inertial sensor data was used, we further explore the possibility to extract the necessary information for consistent 3D mapping over larger areas from images only. In particular, we discuss our recent work focusing on estimating the absolute scale of motion from images as well as finding intersections where the camera path crosses itself to effectively close loops in the mapping process. For this purpose we introduce viewpoint-invariant patches (VIP) as a new 3D feature that we extract from 3D models locally computed from the video sequence. These 3D features have important advantages with respect to traditional 2D SIFT features such as much stronger viewpoint-invariance, a relative pose hypothesis from a single match and a hierarchical matching scheme naturally robust to repetitive structures. In addition, we also briefly discuss some additional work related to absolute scale estimation and multi-camera calibration.