Semi-supervised semantic segmentation needs strong, varied perturbations

TL;DR

This paper investigates the challenges of semi-supervised semantic segmentation, highlighting the importance of strong, varied augmentations like CutOut and CutMix to achieve state-of-the-art results, and proposes segmentation as a benchmark for semi-supervised regularizers.

Contribution

It identifies the lack of low-density class regions in segmentation data as a key challenge and demonstrates that adapted augmentation techniques significantly improve semi-supervised segmentation performance.

Findings

Adapted CutOut and CutMix yield state-of-the-art results.

Segmentation lacks low-density class regions, explaining semi-supervised difficulties.

Segmentation serves as an effective benchmark for semi-supervised regularizers.

Abstract

Consistency regularization describes a class of approaches that have yielded ground breaking results in semi-supervised classification problems. Prior work has established the cluster assumption - under which the data distribution consists of uniform class clusters of samples separated by low density regions - as important to its success. We analyze the problem of semantic segmentation and find that its' distribution does not exhibit low density regions separating classes and offer this as an explanation for why semi-supervised segmentation is a challenging problem, with only a few reports of success. We then identify choice of augmentation as key to obtaining reliable performance without such low-density regions. We find that adapted variants of the recently proposed CutOut and CutMix augmentation techniques yield state-of-the-art semi-supervised semantic segmentation results in standard datasets. Furthermore, given its challenging nature we propose that semantic segmentation acts as an effective acid test for evaluating semi-supervised regularizers. Implementation at: https://github.com/Britefury/cutmix-semisup-seg.