Class Re-Activation Maps for Weakly-Supervised Semantic Segmentation

• Published in: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 959 - 968
• Main Authors: Chen, Zhaozheng, Wang, Tan, Wu, Xiongwei, Hua, Xian-Sheng, Zhang, Hanwang, Sun, Qianru
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.06.2022
• Subjects: Benchmark testing; categorization; Codes; Computational modeling; Computer vision; Feature extraction; grouping and shape analysis; Recognition: detection; retrieval; Segmentation; Semantics; Shape
• ISSN: 2575-7075
• DOI: 10.1109/CVPR52688.2022.00104

Extracting class activation maps (CAM) is arguably the most standard step of generating pseudo masks for weakly-supervised semantic segmentation (WSSS). Yet, we find that the crux of the unsatisfactory pseudo masks is the binary crossentropy loss (BCE) widely used in CAM. Specifically, due to the sum-over-class pooling nature of BCE, each pixel in CAM may be responsive to multiple classes co-occurring in the same receptive field. As a result, given a class, its hot CAM pixels may wrongly invade the area belonging to other classes, or the non-hot ones may be actually a part of the class. To this end, we introduce an embarrassingly simple yet surprisingly effective method: Reactivating the converged CAM with BCE by using softmax crossentropy loss (SCE), dubbed ReCAM. Given an image, we use CAM to extract the feature pixels of each single class, and use them with the class label to learn another fully-connected layer (after the backbone) with SCE. Once converged, we extract ReCAM in the same way as in CAM. Thanks to the contrastive nature of SCE, the pixel response is disentangled into different classes and hence less mask ambiguity is expected. The evaluation on both PASCAL VOC and MS COCO shows that ReCAM not only generates high-quality masks, but also supports plug-and-play in any CAM variant with little overhead. Our code is public at https://github.com/zhaozhengChenIReCAM.