Self-Ensembling with GAN-based Data Augmentation for Domain Adaptation in Semantic Segmentation

TL;DR

This paper introduces a novel framework combining GAN-based data augmentation with self-ensembling to improve unsupervised domain adaptation in semantic segmentation, effectively reducing domain gaps and outperforming existing methods.

Contribution

It proposes a new GAN-based data augmentation technique integrated with self-ensembling for better domain adaptation in semantic segmentation tasks.

Findings

Outperforms state-of-the-art methods on benchmark datasets.

GAN-based augmentation effectively reduces domain gap.

Self-ensembling enhances segmentation accuracy on target domain.

Abstract

Deep learning-based semantic segmentation methods have an intrinsic limitation that training a model requires a large amount of data with pixel-level annotations. To address this challenging issue, many researchers give attention to unsupervised domain adaptation for semantic segmentation. Unsupervised domain adaptation seeks to adapt the model trained on the source domain to the target domain. In this paper, we introduce a self-ensembling technique, one of the successful methods for domain adaptation in classification. However, applying self-ensembling to semantic segmentation is very difficult because heavily-tuned manual data augmentation used in self-ensembling is not useful to reduce the large domain gap in the semantic segmentation. To overcome this limitation, we propose a novel framework consisting of two components, which are complementary to each other. First, we present a data augmentation method based on Generative Adversarial Networks (GANs), which is computationally efficient and effective to facilitate domain alignment. Given those augmented images, we apply self-ensembling to enhance the performance of the segmentation network on the target domain. The proposed method outperforms state-of-the-art semantic segmentation methods on unsupervised domain adaptation benchmarks.