SPIGAN: Privileged Adversarial Learning from Simulation

TL;DR

SPIGAN is an unsupervised domain adaptation method that leverages simulator privileged information and GANs to improve semantic segmentation performance on real-world datasets using synthetic data.

Contribution

It introduces a novel approach combining privileged information from simulators with adversarial learning for domain adaptation.

Findings

Outperforms no adaptation baseline.

Surpasses existing unsupervised domain adaptation methods.

Effective on Cityscapes and Vistas datasets.

Abstract

Deep Learning for Computer Vision depends mainly on the source of supervision.Photo-realistic simulators can generate large-scale automatically labeled syntheticdata, but introduce a domain gap negatively impacting performance. We propose anew unsupervised domain adaptation algorithm, called SPIGAN, relying on Sim-ulator Privileged Information (PI) and Generative Adversarial Networks (GAN).We use internal data from the simulator as PI during the training of a target tasknetwork. We experimentally evaluate our approach on semantic segmentation. Wetrain the networks on real-world Cityscapes and Vistas datasets, using only unla-beled real-world images and synthetic labeled data with z-buffer (depth) PI fromthe SYNTHIA dataset. Our method improves over no adaptation and state-of-the-art unsupervised domain adaptation techniques.