Adversarial Learning and Self-Teaching Techniques for Domain Adaptation in Semantic Segmentation

TL;DR

This paper introduces a novel unsupervised domain adaptation method for semantic segmentation that combines supervised, adversarial, and self-teaching components to improve performance on real-world data using synthetic training data.

Contribution

It proposes a new UDA strategy integrating adversarial learning and self-teaching with class-weighted region growing for better domain adaptation in semantic segmentation.

Findings

Effective adaptation from synthetic to real datasets.

Improved segmentation accuracy on real-world urban scenes.

Enhanced performance on less common classes through class-weighting.

Abstract

Deep learning techniques have been widely used in autonomous driving systems for the semantic understanding of urban scenes. However, they need a huge amount of labeled data for training, which is difficult and expensive to acquire. A recently proposed workaround is to train deep networks using synthetic data, but the domain shift between real world and synthetic representations limits the performance. In this work, a novel Unsupervised Domain Adaptation (UDA) strategy is introduced to solve this issue. The proposed learning strategy is driven by three components: a standard supervised learning loss on labeled synthetic data; an adversarial learning module that exploits both labeled synthetic data and unlabeled real data; finally, a self-teaching strategy applied to unlabeled data. The last component exploits a region growing framework guided by the segmentation confidence. Furthermore, we weighted this component on the basis of the class frequencies to enhance the performance on less common classes. Experimental results prove the effectiveness of the proposed strategy in adapting a segmentation network trained on synthetic datasets, like GTA5 and SYNTHIA, to real world datasets like Cityscapes and Mapillary.