Geometric Unsupervised Domain Adaptation for Semantic Segmentation

TL;DR

This paper introduces GUDA, a novel geometric self-supervised approach that enhances synthetic-to-real domain adaptation for semantic segmentation by leveraging monocular depth estimation, achieving state-of-the-art results.

Contribution

The paper presents a new multi-task framework combining semantic and geometric tasks to improve unsupervised domain adaptation for semantic segmentation.

Findings

GUDA outperforms existing methods on three benchmarks.

The approach scales well with synthetic data quality and quantity.

It also improves depth prediction accuracy.

Abstract

Simulators can efficiently generate large amounts of labeled synthetic data with perfect supervision for hard-to-label tasks like semantic segmentation. However, they introduce a domain gap that severely hurts real-world performance. We propose to use self-supervised monocular depth estimation as a proxy task to bridge this gap and improve sim-to-real unsupervised domain adaptation (UDA). Our Geometric Unsupervised Domain Adaptation method (GUDA) learns a domain-invariant representation via a multi-task objective combining synthetic semantic supervision with real-world geometric constraints on videos. GUDA establishes a new state of the art in UDA for semantic segmentation on three benchmarks, outperforming methods that use domain adversarial learning, self-training, or other self-supervised proxy tasks. Furthermore, we show that our method scales well with the quality and quantity of synthetic data while also improving depth prediction.