RVD-SLAM: A Real-Time Visual SLAM Toward Dynamic Environments Based on Sparsely Semantic Segmentation and Outlier Prior

• Published in: IEEE sensors journal Vol. 23; no. 24; pp. 30773 - 30785
• Main Authors: Zhou, Yao, Tao, Fazhan, Fu, Zhumu, Zhu, Longlong, Ma, Haoxiang
• Format: Journal Article
• Language: English
• Published: New York The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 15.12.2023
• Subjects: Cameras; Data analysis; Localization; Matching; Object recognition; Outliers (statistics); Real time; Semantic segmentation; Semantics; Simultaneous localization and mapping; Tracking
• ISSN: 1530-437X; 1558-1748
• DOI: 10.1109/JSEN.2023.3329123

In visual simultaneous localization and mapping (vSLAM) systems, the rigid assumption of the environment is usually overcome by introducing geometry-based or learning-based methods. However, these methods either rely heavily on depth information or tend to suffer from poor real-time performance. In this article, we present real-time vSLAM (RVD-SLAM), a real-time simultaneous localization and mapping (SLAM) in dynamic environments for red green blue-depth (RGB-D)/stereo/monocular sensors based on sparsely semantic segmentation and outlier prior. RVD-SLAM utilizes feature matching information, which does not rely on depth, to detect outliers and dynamic features. First, an affine consistency constraint is proposed and seamlessly integrated into the matching optimization of the system to efficiently detect outliers. Second, we skillfully combine affine information with the efficiently semantic model, which enables the reuse of previously obtained information, to detect dynamic objects. Finally, to adapt to slightly dynamic environments and reduce the computational cost, the ratio of outliers in the matching set is employed as prior information for sparsely using the semantic method. Moreover, an improved map update method considering dynamic features is designed to ensure the effectiveness of the map used for long-term tracking. The wide experiments on the public TUM and KITTI datasets demonstrate quantitatively that our method improves camera tracking compared to state-of-the-art (SOTA) methods and runs in real-time. To further evaluate RVD-SLAM’s generalization capabilities and performance of dynamic object detection, we also test it in real dynamic scenarios by using handheld RGB-D and stereo cameras.