Boosting Weakly-Supervised Image Segmentation via Representation, Transform, and Compensator

TL;DR

This paper introduces a novel single-stage weakly-supervised image segmentation method using contrastive learning and cross-representation refinement, significantly improving pseudo-mask quality and outperforming state-of-the-art methods on PASCAL VOC 2012.

Contribution

The paper proposes a new single-stage WSIS approach with a siamese network, contrastive learning, and cross-transform regularization to enhance CAM quality and segmentation accuracy.

Findings

Achieves 67.2% mIoU on PASCAL VOC 2012 val set

Outperforms existing WSIS methods in segmentation accuracy

Extends effectively to weakly supervised object localization

Abstract

Weakly-supervised image segmentation (WSIS) is a critical task in computer vision that relies on image-level class labels. Multi-stage training procedures have been widely used in existing WSIS approaches to obtain high-quality pseudo-masks as ground-truth, resulting in significant progress. However, single-stage WSIS methods have recently gained attention due to their potential for simplifying training procedures, despite often suffering from low-quality pseudo-masks that limit their practical applications. To address this issue, we propose a novel single-stage WSIS method that utilizes a siamese network with contrastive learning to improve the quality of class activation maps (CAMs) and achieve a self-refinement process. Our approach employs a cross-representation refinement method that expands reliable object regions by utilizing different feature representations from the backbone. Additionally, we introduce a cross-transform regularization module that learns robust class prototypes for contrastive learning and captures global context information to feed back rough CAMs, thereby improving the quality of CAMs. Our final high-quality CAMs are used as pseudo-masks to supervise the segmentation result. Experimental results on the PASCAL VOC 2012 dataset demonstrate that our method significantly outperforms other state-of-the-art methods, achieving 67.2% and 68.76% mIoU on PASCAL VOC 2012 val set and test set, respectively. Furthermore, our method has been extended to weakly supervised object localization task, and experimental results demonstrate that our method continues to achieve very competitive results.