Class-Balanced Pixel-Level Self-Labeling for Domain Adaptive Semantic Segmentation

• Published in: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 11583 - 11593
• Main Authors: Li, Ruihuang, Li, Shuai, He, Chenhang, Zhang, Yabin, Jia, Xu, Zhang, Lei
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.06.2022
• Subjects: grouping and shape analysis; Self-& semi-& meta- Transfer/low-shot/long-tail learning; Image segmentation; Pipelines; Predictive models; Segmentation; Semantics; Shape; Statistical analysis; Training
• ISSN: 2575-7075
• DOI: 10.1109/CVPR52688.2022.01130

Domain adaptive semantic segmentation aims to learn a model with the supervision of source domain data, and produce satisfactory dense predictions on unlabeled target domain. One popular solution to this challenging task is self-training, which selects high-scoring predictions on target samples as pseudo labels for training. However, the produced pseudo labels often contain much noise because the model is biased to source domain as well as majority categories. To address the above issues, we propose to di-rectly explore the intrinsic pixel distributions of target do-main data, instead of heavily relying on the source domain. Specifically, we simultaneously cluster pixels and rectify pseudo labels with the obtained cluster assignments. This process is done in an online fashion so that pseudo labels could co-evolve with the segmentation model without extra training rounds. To overcome the class imbalance problem on long-tailed categories, we employ a distribution align-ment technique to enforce the marginal class distribution of cluster assignments to be close to that of pseudo labels. The proposed method, namely Class-balanced Pixel-level Self-Labeling (CPSL), improves the segmentation performance on target domain over state-of-the-arts by a large margin, especially on long-tailed categories. The source code is available at ht tps: / / gi thub. com/lslrh/CPSL.