A survey on deep learning-based precise boundary recovery of semantic segmentation for images and point clouds

• Published in: International journal of applied earth observation and geoinformation Vol. 102; p. 102411
• Main Authors: Zhang, Rui, Li, Guangyun, Wunderlich, Thomas, Wang, Li
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.10.2021; Elsevier
• Subjects: 2D images; 3D point clouds; DCNNs; Precise boundary recovery; Semantic segmentation; DCNNs; 2D images; 3D point clouds; Semantic segmentation; Precise boundary recovery
• ISSN: 1569-8432; 1872-826X
• DOI: 10.1016/j.jag.2021.102411

•A comprehensive review of deep learning-based precise boundary recovery techniques for semantic segmentation for 2D images and 3D point clouds.•Fusion of two data types: 2D images and 3D point clouds, rather than only one type or other types.•tatistical analysis of benchmark datasets. Histograms, line charts and scatter charts are designed to compare and analyze the five public 2D image datasets according to five indicators.•Comparison and analysis between the initial semantic segmentation results and the results after boundary recovery. Precise localization of semantic segmentation is attracting increasing attention, and salient performances are dominated by deep learning-based methods, especially deep convolutional neural networks (DCNNs). However, the outputs from the final layer of DCNNs are not sufficiently localized for accurate object boundaries due to their invariance properties, which makes precise boundary recovery of semantic segmentation an academically challenging question. Both 2D and 3D objects suffer from the same problem. Considering this, this paper conducts a comprehensive survey of precise boundary recovery for semantic segmentation, focusing mainly on 2D images and 3D point clouds. Firstly, we formulate the problem of potential boundary recovery for semantic segmentation based on DCNNs, elaborate on the terminology as well as background concepts in this field. Then, we categorize boundary recovery methods into four strategies according to their techniques and network architectures to discuss how they obtain accurate boundaries of semantic segmentation. Next, publicly available datasets on which they have been assessed are argued. To compare these datasets, we design diagrams based on five indicators to help researchers judge which are the ones that best suit their tasks. Moreover, we further compare and analyze the performance of all the reviewed methods through experimental results. Finally, current challenges and prospective research issues are discussed extensively.