MuSCLe: A Multi-Strategy Contrastive Learning Framework for Weakly Supervised Semantic Segmentation

TL;DR

MuSCLe is a novel contrastive learning framework that enhances feature representations in weakly supervised semantic segmentation by leveraging multiple strategies at various levels, leading to improved performance on benchmark datasets.

Contribution

The paper introduces MuSCLe, a multi-strategy contrastive learning framework that significantly improves weakly supervised semantic segmentation by exploiting sample similarities and dissimilarities at multiple levels.

Findings

Outperforms state-of-the-art on PASCAL VOC 2012

Enhances feature representations in WSSS

Effective at image, region, pixel, and boundary levels

Abstract

Weakly supervised semantic segmentation (WSSS) has gained significant popularity since it relies only on weak labels such as image level annotations rather than pixel level annotations required by supervised semantic segmentation (SSS) methods. Despite drastically reduced annotation costs, typical feature representations learned from WSSS are only representative of some salient parts of objects and less reliable compared to SSS due to the weak guidance during training. In this paper, we propose a novel Multi-Strategy Contrastive Learning (MuSCLe) framework to obtain enhanced feature representations and improve WSSS performance by exploiting similarity and dissimilarity of contrastive sample pairs at image, region, pixel and object boundary levels. Extensive experiments demonstrate the effectiveness of our method and show that MuSCLe outperforms the current state-of-the-art on the widely used PASCAL VOC 2012 dataset.