TWIST: Two-Way Inter-label Self-Training for Semi-supervised 3D Instance Segmentation

We explore the way to alleviate the label-hungry problem in a semi-supervised setting for 3D instance segmentation. To leverage the unlabeled data to boost model performance, we present a novel Two-Way Inter-label Self-Training framework named TWIST. It exploits inherent correlations between semanti...

Full description

Saved in:
Bibliographic Details
Published in2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 1090 - 1099
Main Authors Chu, Ruihang, Ye, Xiaoqing, Liu, Zhengzhe, Tan, Xiao, Qi, Xiaojuan, Fu, Chi-Wing, Jia, Jiaya
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.06.2022
Subjects
categorization
Correlation
Data models
Noise reduction
Pattern recognition
Recognition: detection
retrieval; 3D from multi-view and sensors; Self-& semi-& meta- & unsupervised learning
Semantics
Three-dimensional displays
Training
Online AccessGet full text

Cover

More Information
Summary:We explore the way to alleviate the label-hungry problem in a semi-supervised setting for 3D instance segmentation. To leverage the unlabeled data to boost model performance, we present a novel Two-Way Inter-label Self-Training framework named TWIST. It exploits inherent correlations between semantic understanding and instance information of a scene. Specifically, we consider two kinds of pseudo labels for semantic- and instance-level supervision. Our key design is to provide object-level information for denoising pseudo labels and make use of their correlation for two-way mutual enhancement, thereby iteratively promoting the pseudo-label qualities. TWIST attains leading performance on both ScanNet and S3DIS, compared to recent 3D pre-training approaches, and can cooperate with them to further enhance performance, e.g., +4.4% AP 50 on 1%-label ScanNet data-efficient benchmark. Code is available at https://github.com/dvlab-research/TWIST.
ISSN:2575-7075
DOI:10.1109/CVPR52688.2022.00117