Masked Collaborative Contrast for Weakly Supervised Semantic Segmentation

TL;DR

This paper proposes Masked Collaborative Contrast (MCC), a novel weakly supervised semantic segmentation method that leverages masked image modeling and contrastive learning to better highlight semantic regions and improve segmentation accuracy.

Contribution

The paper introduces MCC, a new framework that uses neighborhood relations and contrastive learning with masked local and global outputs for weakly supervised segmentation.

Findings

MCC effectively aligns global and local image features.

The method achieves state-of-the-art performance on standard datasets.

Source code is publicly available for reproducibility.

Abstract

This study introduces an efficacious approach, Masked Collaborative Contrast (MCC), to highlight semantic regions in weakly supervised semantic segmentation. MCC adroitly draws inspiration from masked image modeling and contrastive learning to devise a novel framework that induces keys to contract toward semantic regions. Unlike prevalent techniques that directly eradicate patch regions in the input image when generating masks, we scrutinize the neighborhood relations of patch tokens by exploring masks considering keys on the affinity matrix. Moreover, we generate positive and negative samples in contrastive learning by utilizing the masked local output and contrasting it with the global output. Elaborate experiments on commonly employed datasets evidences that the proposed MCC mechanism effectively aligns global and local perspectives within the image, attaining impressive performance. The source code is available at \url{https://github.com/fwu11/MCC}.