Weakly Supervised Segmentation on Outdoor 4D point clouds with Temporal Matching and Spatial Graph Propagation

• Published in: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 11830 - 11839
• Main Authors: Shi, Hanyu, Wei, Jiacheng, Li, Ruibo, Liu, Fayao, Lin, Guosheng
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.06.2022
• Subjects: Annotations; Computational modeling; Computer vision; grouping and shape analysis; 3D from multi-view and sensors; Image and video synthesis and generation; Scene analysis and understanding; Self-& semi-& meta- Video analysis and understanding; Point cloud compression; Segmentation; Solid modeling; Three-dimensional displays; Training
• ISSN: 2575-7075
• DOI: 10.1109/CVPR52688.2022.01154

Existing point cloud segmentation methods require a large amount of annotated data, especially for the outdoor point cloud scene. Due to the complexity of the outdoor 3D scenes, manual annotations on the outdoor point cloud scene are time-consuming and expensive. In this paper, we study how to achieve scene understanding with limited annotated data. Treating 100 consecutive frames as a sequence, we divide the whole dataset into a series of sequences and annotate only 0.1% points in the first frame of each sequence to reduce the annotation requirements. This leads to a total annotation budget of 0.001%. We propose a novel temporal-spatial framework for effective weakly supervised learning to generate high-quality pseudo labels from these limited annotated data. Specifically, the frame-work contains two modules: an matching module in temporal dimension to propagate pseudo labels across different frames, and a graph propagation module in spatial dimension to propagate the information of pseudo labels to the entire point clouds in each frame. With only 0.001% annotations for training, experimental results on both SemanticKITTI and SemanticPOSS shows our weakly supervised two-stage framework is comparable to some existing fully supervised methods. We also evaluate our framework with 0.005% initial annotations on SemanticKITTI, and achieve a result close to fully supervised backbone model.