Semi-supervised Semantic Segmentation with Error Localization Network

TL;DR

This paper introduces an Error Localization Network (ELN) for semi-supervised semantic segmentation, effectively identifying and ignoring incorrect pseudo labels to improve training robustness and outperform existing methods on benchmark datasets.

Contribution

The paper proposes a novel ELN module that localizes pseudo label errors and a training strategy to improve its generalization, enhancing semi-supervised segmentation performance.

Findings

Outperforms existing methods on PASCAL VOC 2012

Outperforms existing methods on Cityscapes

Robust against pseudo label noise

Abstract

This paper studies semi-supervised learning of semantic segmentation, which assumes that only a small portion of training images are labeled and the others remain unlabeled. The unlabeled images are usually assigned pseudo labels to be used in training, which however often causes the risk of performance degradation due to the confirmation bias towards errors on the pseudo labels. We present a novel method that resolves this chronic issue of pseudo labeling. At the heart of our method lies error localization network (ELN), an auxiliary module that takes an image and its segmentation prediction as input and identifies pixels whose pseudo labels are likely to be wrong. ELN enables semi-supervised learning to be robust against inaccurate pseudo labels by disregarding label noises during training and can be naturally integrated with self-training and contrastive learning. Moreover, we introduce a new learning strategy for ELN that simulates plausible and diverse segmentation errors during training of ELN to enhance its generalization. Our method is evaluated on PASCAL VOC 2012 and Cityscapes, where it outperforms all existing methods in every evaluation setting.