Using Unreliable Pseudo-Labels for Label-Efficient Semantic Segmentation

TL;DR

This paper introduces a novel method for semantic segmentation that utilizes unreliable pseudo-labels by treating uncertain pixels as negatives for unlikely classes, improving label efficiency and model training.

Contribution

It proposes a new pipeline that leverages all pixels, including unreliable ones, by categorizing them as negatives, with adaptive thresholding for better training.

Findings

Outperforms state-of-the-art methods on multiple benchmarks.

Effectively uses unreliable pixels to enhance training.

Adaptive thresholding improves pseudo-label quality.

Abstract

The crux of label-efficient semantic segmentation is to produce high-quality pseudo-labels to leverage a large amount of unlabeled or weakly labeled data. A common practice is to select the highly confident predictions as the pseudo-ground-truths for each pixel, but it leads to a problem that most pixels may be left unused due to their unreliability. However, we argue that every pixel matters to the model training, even those unreliable and ambiguous pixels. Intuitively, an unreliable prediction may get confused among the top classes, however, it should be confident about the pixel not belonging to the remaining classes. Hence, such a pixel can be convincingly treated as a negative key to those most unlikely categories. Therefore, we develop an effective pipeline to make sufficient use of unlabeled data. Concretely, we separate reliable and unreliable pixels via the entropy of predictions, push each unreliable pixel to a category-wise queue that consists of negative keys, and manage to train the model with all candidate pixels. Considering the training evolution, we adaptively adjust the threshold for the reliable-unreliable partition. Experimental results on various benchmarks and training settings demonstrate the superiority of our approach over the state-of-the-art alternatives.