Discovering Latent Classes for Semi-Supervised Semantic Segmentation

TL;DR

This paper introduces a semi-supervised semantic segmentation method that learns latent classes to leverage unlabeled data, significantly reducing annotation costs while achieving state-of-the-art results.

Contribution

It proposes a novel approach to learn latent classes related to semantic classes, improving semi-supervised segmentation performance.

Findings

Achieves state-of-the-art results on Pascal VOC and Cityscapes.

Latent classes learned have intuitive semantic meanings.

Effectively leverages unlabeled images for segmentation.

Abstract

High annotation costs are a major bottleneck for the training of semantic segmentation systems. Therefore, methods working with less annotation effort are of special interest. This paper studies the problem of semi-supervised semantic segmentation. This means that only a small subset of the training images is annotated while the other training images do not contain any annotation. In order to leverage the information present in the unlabeled images, we propose to learn a second task that is related to semantic segmentation but easier. On labeled images, we learn latent classes consistent with semantic classes so that the variety of semantic classes assigned to a latent class is as low as possible. On unlabeled images, we predict a probability map for latent classes and use it as a supervision signal to learn semantic segmentation. The latent classes, as well as the semantic classes, are simultaneously predicted by a two-branch network. In our experiments on Pascal VOC and Cityscapes, we show that the latent classes learned this way have an intuitive meaning and that the proposed method achieves state of the art results for semi-supervised semantic segmentation.