Classes Matter: A Fine-grained Adversarial Approach to Cross-domain Semantic Segmentation

TL;DR

This paper introduces a fine-grained adversarial learning approach for cross-domain semantic segmentation that aligns features at the class level, leading to improved performance over traditional holistic methods.

Contribution

The paper proposes a novel class-level feature alignment method using a fine-grained adversarial strategy and domain encodings, enhancing domain adaptation in semantic segmentation.

Findings

Achieves better class-level alignment than state-of-the-art methods.

Significantly improves performance on multiple domain adaptation benchmarks.

Outperforms global and class-wise alignment methods.

Abstract

Despite great progress in supervised semantic segmentation,a large performance drop is usually observed when deploying the model in the wild. Domain adaptation methods tackle the issue by aligning the source domain and the target domain. However, most existing methods attempt to perform the alignment from a holistic view, ignoring the underlying class-level data structure in the target domain. To fully exploit the supervision in the source domain, we propose a fine-grained adversarial learning strategy for class-level feature alignment while preserving the internal structure of semantics across domains. We adopt a fine-grained domain discriminator that not only plays as a domain distinguisher, but also differentiates domains at class level. The traditional binary domain labels are also generalized to domain encodings as the supervision signal to guide the fine-grained feature alignment. An analysis with Class Center Distance (CCD) validates that our fine-grained adversarial strategy achieves better class-level alignment compared to other state-of-the-art methods. Our method is easy to implement and its effectiveness is evaluated on three classical domain adaptation tasks, i.e., GTA5 to Cityscapes, SYNTHIA to Cityscapes and Cityscapes to Cross-City. Large performance gains show that our method outperforms other global feature alignment based and class-wise alignment based counterparts. The code is publicly available at https://github.com/JDAI-CV/FADA.