Progressive Proxy Anchor Propagation for Unsupervised Semantic Segmentation

TL;DR

This paper introduces PPAP, a progressive strategy that improves unsupervised semantic segmentation by iteratively refining positive sample selection and managing ambiguous regions, leading to state-of-the-art results.

Contribution

The paper proposes a novel Progressive Proxy Anchor Propagation method that enhances patch-level supervision in unsupervised segmentation by dynamic proxy relocation and ambiguous zone handling.

Findings

Achieves state-of-the-art performance on multiple datasets.

Effectively refines positive sample selection through proxy relocation.

Improves negative set reliability by excluding ambiguous samples.

Abstract

The labor-intensive labeling for semantic segmentation has spurred the emergence of Unsupervised Semantic Segmentation. Recent studies utilize patch-wise contrastive learning based on features from image-level self-supervised pretrained models. However, relying solely on similarity-based supervision from image-level pretrained models often leads to unreliable guidance due to insufficient patch-level semantic representations. To address this, we propose a Progressive Proxy Anchor Propagation (PPAP) strategy. This method gradually identifies more trustworthy positives for each anchor by relocating its proxy to regions densely populated with semantically similar samples. Specifically, we initially establish a tight boundary to gather a few reliable positive samples around each anchor. Then, considering the distribution of positive samples, we relocate the proxy anchor towards areas with a higher concentration of positives and adjust the positiveness boundary based on the propagation degree of the proxy anchor. Moreover, to account for ambiguous regions where positive and negative samples may coexist near the positiveness boundary, we introduce an instance-wise ambiguous zone. Samples within these zones are excluded from the negative set, further enhancing the reliability of the negative set. Our state-of-the-art performances on various datasets validate the effectiveness of the proposed method for Unsupervised Semantic Segmentation.