Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition

• Published in: arXiv.org
• Main Authors: Zi-Hao Zhou, Fang, Siyuan, Zi-Jing Zhou, Tong, Wei, Wan, Yuanyu, Min-Ling, Zhang
• Format: Paper
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 08.10.2024
• Subjects: Datasets; Estimation; Labels; Neural networks; Semi-supervised learning; Source code
• ISSN: 2331-8422

Long-tailed semi-supervised learning poses a significant challenge in training models with limited labeled data exhibiting a long-tailed label distribution. Current state-of-the-art LTSSL approaches heavily rely on high-quality pseudo-labels for large-scale unlabeled data. However, these methods often neglect the impact of representations learned by the neural network and struggle with real-world unlabeled data, which typically follows a different distribution than labeled data. This paper introduces a novel probabilistic framework that unifies various recent proposals in long-tail learning. Our framework derives the class-balanced contrastive loss through Gaussian kernel density estimation. We introduce a continuous contrastive learning method, CCL, extending our framework to unlabeled data using reliable and smoothed pseudo-labels. By progressively estimating the underlying label distribution and optimizing its alignment with model predictions, we tackle the diverse distribution of unlabeled data in real-world scenarios. Extensive experiments across multiple datasets with varying unlabeled data distributions demonstrate that CCL consistently outperforms prior state-of-the-art methods, achieving over 4% improvement on the ImageNet-127 dataset. Our source code is available at https://github.com/zhouzihao11/CCL