Saliency Guided Inter- and Intra-Class Relation Constraints for Weakly Supervised Semantic Segmentation

TL;DR

This paper introduces a saliency guided relation constraint framework to improve weakly supervised semantic segmentation by expanding object regions in class activation maps, leading to more accurate pseudo labels.

Contribution

The proposed I$^2$CRC framework effectively enhances CAM-based pseudo labels through inter- and intra-class relation constraints and saliency guidance, improving segmentation accuracy.

Findings

Outperforms state-of-the-art methods on PASCAL VOC 2012 and COCO datasets.

Significantly improves object region activation in CAMs.

Achieves higher segmentation accuracy with refined pseudo labels.

Abstract

Weakly supervised semantic segmentation with only image-level labels aims to reduce annotation costs for the segmentation task. Existing approaches generally leverage class activation maps (CAMs) to locate the object regions for pseudo label generation. However, CAMs can only discover the most discriminative parts of objects, thus leading to inferior pixel-level pseudo labels. To address this issue, we propose a saliency guided Inter- and Intra-Class Relation Constrained (I$^2$CRC) framework to assist the expansion of the activated object regions in CAMs. Specifically, we propose a saliency guided class-agnostic distance module to pull the intra-category features closer by aligning features to their class prototypes. Further, we propose a class-specific distance module to push the inter-class features apart and encourage the object region to have a higher activation than the background. Besides strengthening the capability of the classification network to activate more integral object regions in CAMs, we also introduce an object guided label refinement module to take a full use of both the segmentation prediction and the initial labels for obtaining superior pseudo-labels. Extensive experiments on PASCAL VOC 2012 and COCO datasets demonstrate well the effectiveness of I$^2$CRC over other state-of-the-art counterparts. The source codes, models, and data have been made available at \url{https://github.com/NUST-Machine-Intelligence-Laboratory/I2CRC}.