Domain Adaptive Semantic Segmentation Using Weak Labels

TL;DR

This paper introduces a novel framework for domain adaptive semantic segmentation that leverages weak labels, either from model predictions or human annotations, to improve feature alignment and segmentation accuracy across domains.

Contribution

It proposes a new weak-label based domain adaptation framework that combines feature alignment and pseudo-labeling for semantic segmentation.

Findings

Significant improvements over state-of-the-art in unsupervised domain adaptation.

Establishment of a new benchmark for weakly-supervised domain adaptation.

Effective category-wise domain alignment using weak labels.

Abstract

Learning semantic segmentation models requires a huge amount of pixel-wise labeling. However, labeled data may only be available abundantly in a domain different from the desired target domain, which only has minimal or no annotations. In this work, we propose a novel framework for domain adaptation in semantic segmentation with image-level weak labels in the target domain. The weak labels may be obtained based on a model prediction for unsupervised domain adaptation (UDA), or from a human annotator in a new weakly-supervised domain adaptation (WDA) paradigm for semantic segmentation. Using weak labels is both practical and useful, since (i) collecting image-level target annotations is comparably cheap in WDA and incurs no cost in UDA, and (ii) it opens the opportunity for category-wise domain alignment. Our framework uses weak labels to enable the interplay between feature alignment and pseudo-labeling, improving both in the process of domain adaptation. Specifically, we develop a weak-label classification module to enforce the network to attend to certain categories, and then use such training signals to guide the proposed category-wise alignment method. In experiments, we show considerable improvements with respect to the existing state-of-the-arts in UDA and present a new benchmark in the WDA setting. Project page is at http://www.nec-labs.com/~mas/WeakSegDA.