FaSS-MVS: Fast Multi-View Stereo with Surface-Aware Semi-Global Matching from UAV-Borne Monocular Imagery

• Published in: Sensors (Basel, Switzerland) Vol. 24; no. 19; p. 6397
• Main Authors: Ruf, Boitumelo, Weinmann, Martin, Hinz, Stefan
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI AG 02.10.2024; MDPI
• Subjects: Algorithms; Augmented reality; Building facades; Cameras; Computer vision; Datasets; Digital map services; Drone aircraft; Geometry; Localization; Mapping; multi-view stereo; oblique aerial imagery; online processing; Photogrammetry; plane-sweep multi-image matching; Remote sensing; semi-global optimization; Software; surface-awareness; Germany; online processing; oblique aerial imagery; semi-global optimization; plane-sweep multi-image matching; multi-view stereo; UAVs; surface-awareness
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s24196397

With FaSS-MVS, we present a fast, surface-aware semi-global optimization approach for multi-view stereo that allows for rapid depth and normal map estimation from monocular aerial video data captured by unmanned aerial vehicles (UAVs). The data estimated by FaSS-MVS, in turn, facilitate online 3D mapping, meaning that a 3D map of the scene is immediately and incrementally generated as the image data are acquired or being received. FaSS-MVS is composed of a hierarchical processing scheme in which depth and normal data, as well as corresponding confidence scores, are estimated in a coarse-to-fine manner, allowing efficient processing of large scene depths, such as those inherent in oblique images acquired by UAVs flying at low altitudes. The actual depth estimation uses a plane-sweep algorithm for dense multi-image matching to produce depth hypotheses from which the actual depth map is extracted by means of a surface-aware semi-global optimization, reducing the fronto-parallel bias of Semi-Global Matching (SGM). Given the estimated depth map, the pixel-wise surface normal information is then computed by reprojecting the depth map into a point cloud and computing the normal vectors within a confined local neighborhood. In a thorough quantitative and ablative study, we show that the accuracy of the 3D information computed by FaSS-MVS is close to that of state-of-the-art offline multi-view stereo approaches, with the error not even an order of magnitude higher than that of COLMAP. At the same time, however, the average runtime of FaSS-MVS for estimating a single depth and normal map is less than 14% of that of COLMAP, allowing us to perform online and incremental processing of full HD images at 1–2 Hz.