TridentAdapt: Learning Domain-invariance via Source-Target Confrontation and Self-induced Cross-domain Augmentation

TL;DR

This paper introduces TridentAdapt, a novel architecture and training pipeline that enforce domain-invariance and self-induced augmentation, significantly improving virtual-to-real domain adaptation for semantic segmentation.

Contribution

The paper proposes a trident-like architecture with confrontational constraints and a self-induced augmentation pipeline, advancing domain-invariant feature learning for semantic segmentation.

Findings

Achieves state-of-the-art results on GTA5 to Cityscapes adaptation

Introduces a novel confrontational training approach for domain invariance

Develops a self-induced cross-domain augmentation method

Abstract

Due to the difficulty of obtaining ground-truth labels, learning from virtual-world datasets is of great interest for real-world applications like semantic segmentation. From domain adaptation perspective, the key challenge is to learn domain-agnostic representation of the inputs in order to benefit from virtual data. In this paper, we propose a novel trident-like architecture that enforces a shared feature encoder to satisfy confrontational source and target constraints simultaneously, thus learning a domain-invariant feature space. Moreover, we also introduce a novel training pipeline enabling self-induced cross-domain data augmentation during the forward pass. This contributes to a further reduction of the domain gap. Combined with a self-training process, we obtain state-of-the-art results on benchmark datasets (e.g. GTA5 or Synthia to Cityscapes adaptation). Code and pre-trained models are available at https://github.com/HMRC-AEL/TridentAdapt