PseudoSeg: Designing Pseudo Labels for Semantic Segmentation

TL;DR

PseudoSeg introduces a novel pseudo-labeling approach for semantic segmentation that enhances semi-supervised learning by generating well-calibrated structured pseudo labels, improving performance across data regimes.

Contribution

It presents a network-agnostic pseudo-labeling strategy tailored for structured outputs, addressing challenges in applying SSL to semantic segmentation.

Findings

Effective in low-data and high-data regimes

Well-calibrated pseudo labels improve segmentation accuracy

Diverse source fusion and strong augmentation are crucial

Abstract

Recent advances in semi-supervised learning (SSL) demonstrate that a combination of consistency regularization and pseudo-labeling can effectively improve image classification accuracy in the low-data regime. Compared to classification, semantic segmentation tasks require much more intensive labeling costs. Thus, these tasks greatly benefit from data-efficient training methods. However, structured outputs in segmentation render particular difficulties (e.g., designing pseudo-labeling and augmentation) to apply existing SSL strategies. To address this problem, we present a simple and novel re-design of pseudo-labeling to generate well-calibrated structured pseudo labels for training with unlabeled or weakly-labeled data. Our proposed pseudo-labeling strategy is network structure agnostic to apply in a one-stage consistency training framework. We demonstrate the effectiveness of the proposed pseudo-labeling strategy in both low-data and high-data regimes. Extensive experiments have validated that pseudo labels generated from wisely fusing diverse sources and strong data augmentation are crucial to consistency training for segmentation. The source code is available at https://github.com/googleinterns/wss.