Separate and Conquer: Decoupling Co-occurrence via Decomposition and Representation for Weakly Supervised Semantic Segmentation

TL;DR

This paper introduces a 'Separate and Conquer' scheme for weakly supervised semantic segmentation that decouples co-occurring objects in images and enhances feature representations, leading to improved segmentation accuracy without external supervision.

Contribution

The proposed SeCo method innovatively separates co-occurring objects via image decomposition and improves semantic features through a dual-teacher contrastive architecture, addressing co-occurrence issues in WSSS.

Findings

Outperforms previous methods on PASCAL VOC and MS COCO datasets.

Effectively reduces false activations caused by object co-occurrence.

Streamlines WSSS pipeline end-to-end without external supervision.

Abstract

Weakly supervised semantic segmentation (WSSS) with image-level labels aims to achieve segmentation tasks without dense annotations. However, attributed to the frequent coupling of co-occurring objects and the limited supervision from image-level labels, the challenging co-occurrence problem is widely present and leads to false activation of objects in WSSS. In this work, we devise a 'Separate and Conquer' scheme SeCo to tackle this issue from dimensions of image space and feature space. In the image space, we propose to 'separate' the co-occurring objects with image decomposition by subdividing images into patches. Importantly, we assign each patch a category tag from Class Activation Maps (CAMs), which spatially helps remove the co-context bias and guide the subsequent representation. In the feature space, we propose to 'conquer' the false activation by enhancing semantic representation with multi-granularity knowledge contrast. To this end, a dual-teacher-single-student architecture is designed and tag-guided contrast is conducted, which guarantee the correctness of knowledge and further facilitate the discrepancy among co-contexts. We streamline the multi-staged WSSS pipeline end-to-end and tackle this issue without external supervision. Extensive experiments are conducted, validating the efficiency of our method and the superiority over previous single-staged and even multi-staged competitors on PASCAL VOC and MS COCO. Code is available at https://github.com/zwyang6/SeCo.git.