Semantic and edge-based visual odometry by joint minimizing semantic and edge distance error

• Published in: Image and vision computing Vol. 113; p. 104240
• Main Authors: Peng, Jingquan, Liu, Yanqing, Jiang, Haochen
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.09.2021
• Subjects: Edge alignment; Edge-based visual odometry; Semantic segmentation; Semantic visual odometry distance transform (DT); Semantic visual odometry distance transform (DT); Edge-based visual odometry; Semantic segmentation; Edge alignment
• ISSN: 0262-8856; 1872-8138
• DOI: 10.1016/j.imavis.2021.104240

In recent years, the progress made in deep learning for semantic segmentation has advanced development of semantic visual odometry (VO). Along with point-based and direct methods, VO has recently used edge features. However, mismatches are common in scenes in which the distribution of edges is complex owing to the lack of appropriate descriptors for edges at the present. In this paper, we propose a semantic-segmentation-aided edge-based VO (DSEVO). It is intended to improve the localization accuracy by decreasing mismatches in the edge alignment. In the reprojection process, the semantic and edge distance residual are considered to reduce the mismatches of edges between different frames. Then, camera motion estimation is accomplished by jointly minimizing the semantic and edge cost function. Our proposed method was evaluated on the public VKITTI and TUM RGB-D datasets. It was compared with state-of-the-art methods, including the respective feature-point-based, direct, and edge-based methods. We implemented a semantic-edge-based VO system. The experimental results showed that our method achieved the highest accuracy on most of the testing sequences. •Edge-based methods are investigated in this study and a new method is proposed•Proposed method is first to combine semantic segmentation with edge-based odometry•Semantic-based distance error term is introduced to improve positioning accuracy•Semantic-based distance error and edge-based distance error are joined•Experiments demonstrate the increased accuracy by using the combined approach