Generalize then Adapt: Source-Free Domain Adaptive Semantic Segmentation

TL;DR

This paper introduces a source-free domain adaptation method for semantic segmentation, combining domain generalization and self-training with a novel auto-encoder, outperforming prior methods on standard benchmarks.

Contribution

It proposes a new framework that enables source-free domain adaptation by integrating domain generalization, reliable pseudo-label extraction, and a conditional auto-encoder for improved segmentation.

Findings

Outperforms non-source-free methods on GTA5-to-Cityscapes and SYNTHIA-to-Cityscapes benchmarks.

Demonstrates effectiveness of the auto-encoder in reducing spatial irregularities.

Compatible with online adaptation for sequential environment deployment.

Abstract

Unsupervised domain adaptation (DA) has gained substantial interest in semantic segmentation. However, almost all prior arts assume concurrent access to both labeled source and unlabeled target, making them unsuitable for scenarios demanding source-free adaptation. In this work, we enable source-free DA by partitioning the task into two: a) source-only domain generalization and b) source-free target adaptation. Towards the former, we provide theoretical insights to develop a multi-head framework trained with a virtually extended multi-source dataset, aiming to balance generalization and specificity. Towards the latter, we utilize the multi-head framework to extract reliable target pseudo-labels for self-training. Additionally, we introduce a novel conditional prior-enforcing auto-encoder that discourages spatial irregularities, thereby enhancing the pseudo-label quality. Experiments on the standard GTA5-to-Cityscapes and SYNTHIA-to-Cityscapes benchmarks show our superiority even against the non-source-free prior-arts. Further, we show our compatibility with online adaptation enabling deployment in a sequentially changing environment.