Content Disentanglement for Semantically Consistent Synthetic-to-Real Domain Adaptation

TL;DR

This paper introduces an unsupervised domain adaptation network for synthetic-to-real image transfer in autonomous driving, focusing on maintaining semantic consistency while reducing the domain gap.

Contribution

It proposes a novel content disentanglement approach with shared content encoder and fixed style code for semantically consistent image translation.

Findings

Improved semantic consistency in synthetic-to-real image translation.

Enhanced perception model performance on real data.

Effective domain gap reduction without supervision.

Abstract

Synthetic data generation is an appealing approach to generate novel traffic scenarios in autonomous driving. However, deep learning perception algorithms trained solely on synthetic data encounter serious performance drops when they are tested on real data. Such performance drops are commonly attributed to the domain gap between real and synthetic data. Domain adaptation methods that have been applied to mitigate the aforementioned domain gap achieve visually appealing results, but usually introduce semantic inconsistencies into the translated samples. In this work, we propose a novel, unsupervised, end-to-end domain adaptation network architecture that enables semantically consistent \textit{sim2real} image transfer. Our method performs content disentanglement by employing shared content encoder and fixed style code.