Bootstrapping Semantic Segmentation with Regional Contrast

TL;DR

ReCo is a regional contrastive learning framework that enhances semantic segmentation by focusing on hard negative pixels, improving performance especially in semi-supervised settings with minimal labeled data.

Contribution

ReCo introduces a simple, memory-efficient regional contrastive learning method that boosts semantic segmentation accuracy in both semi-supervised and supervised contexts.

Findings

Achieves high-quality segmentation with only 5 examples per class.

Improves boundary smoothness and convergence speed.

Effective in semi-supervised learning with limited labels.

Abstract

We present ReCo, a contrastive learning framework designed at a regional level to assist learning in semantic segmentation. ReCo performs semi-supervised or supervised pixel-level contrastive learning on a sparse set of hard negative pixels, with minimal additional memory footprint. ReCo is easy to implement, being built on top of off-the-shelf segmentation networks, and consistently improves performance in both semi-supervised and supervised semantic segmentation methods, achieving smoother segmentation boundaries and faster convergence. The strongest effect is in semi-supervised learning with very few labels. With ReCo, we achieve high-quality semantic segmentation models, requiring only 5 examples of each semantic class. Code is available at https://github.com/lorenmt/reco.