Contrastive Prompt Clustering for Weakly Supervised Semantic Segmentation

TL;DR

This paper introduces Contrastive Prompt Clustering (CPC), a novel framework for Weakly Supervised Semantic Segmentation that leverages large language models and contrastive learning to improve class discrimination and boundary accuracy.

Contribution

CPC is the first to integrate LLM-derived category clusters with contrastive loss for enhanced WSSS performance, addressing inter-class similarity issues.

Findings

CPC outperforms state-of-the-art methods on PASCAL VOC 2012.

CPC achieves superior results on MS COCO 2014.

Hierarchical clustering improves fine-grained segmentation boundaries.

Abstract

Weakly Supervised Semantic Segmentation (WSSS) with image-level labels has gained attention for its cost-effectiveness. Most existing methods emphasize inter-class separation, often neglecting the shared semantics among related categories and lacking fine-grained discrimination. To address this, we propose Contrastive Prompt Clustering (CPC), a novel WSSS framework. CPC exploits Large Language Models (LLMs) to derive category clusters that encode intrinsic inter-class relationships, and further introduces a class-aware patch-level contrastive loss to enforce intra-class consistency and inter-class separation. This hierarchical design leverages clusters as coarse-grained semantic priors while preserving fine-grained boundaries, thereby reducing confusion among visually similar categories. Experiments on PASCAL VOC 2012 and MS COCO 2014 demonstrate that CPC surpasses existing state-of-the-art methods in WSSS.