Learning from Synthetic Data: Addressing Domain Shift for Semantic Segmentation

TL;DR

This paper presents a GAN-based method for domain adaptation in semantic segmentation, effectively bridging the gap between synthetic and real images, and demonstrating state-of-the-art results and generalization to unseen domains.

Contribution

It introduces a novel GAN-based approach that aligns feature embeddings across domains, improving segmentation performance without relying on simple adversarial or superpixel methods.

Findings

Achieves state-of-the-art results on synthetic to real domain adaptation tasks.

Generalizes well to unseen domains, indicating robust feature alignment.

Improves the alignment of source and target feature distributions.

Abstract

Visual Domain Adaptation is a problem of immense importance in computer vision. Previous approaches showcase the inability of even deep neural networks to learn informative representations across domain shift. This problem is more severe for tasks where acquiring hand labeled data is extremely hard and tedious. In this work, we focus on adapting the representations learned by segmentation networks across synthetic and real domains. Contrary to previous approaches that use a simple adversarial objective or superpixel information to aid the process, we propose an approach based on Generative Adversarial Networks (GANs) that brings the embeddings closer in the learned feature space. To showcase the generality and scalability of our approach, we show that we can achieve state of the art results on two challenging scenarios of synthetic to real domain adaptation. Additional exploratory experiments show that our approach: (1) generalizes to unseen domains and (2) results in improved alignment of source and target distributions.