Amplitude Spectrum Transformation for Open Compound Domain Adaptive Semantic Segmentation

TL;DR

This paper introduces a novel amplitude spectrum transformation (AST) method for open compound domain adaptation in semantic segmentation, improving domain disentanglement and achieving state-of-the-art results without adversarial training.

Contribution

It proposes a Fourier-based amplitude spectrum transformation (AST) for better domain disentanglement in OCDA, avoiding complex adversarial methods and simplifying adaptation.

Findings

Achieves leading performance on OCDA scene segmentation benchmarks.

Demonstrates effectiveness of amplitude spectrum features for domain adaptation.

Provides a clustering-free, adversarial-free adaptation technique.

Abstract

Open compound domain adaptation (OCDA) has emerged as a practical adaptation setting which considers a single labeled source domain against a compound of multi-modal unlabeled target data in order to generalize better on novel unseen domains. We hypothesize that an improved disentanglement of domain-related and task-related factors of dense intermediate layer features can greatly aid OCDA. Prior-arts attempt this indirectly by employing adversarial domain discriminators on the spatial CNN output. However, we find that latent features derived from the Fourier-based amplitude spectrum of deep CNN features hold a more tractable mapping with domain discrimination. Motivated by this, we propose a novel feature space Amplitude Spectrum Transformation (AST). During adaptation, we employ the AST auto-encoder for two purposes. First, carefully mined source-target instance pairs undergo a simulation of cross-domain feature stylization (AST-Sim) at a particular layer by altering the AST-latent. Second, AST operating at a later layer is tasked to normalize (AST-Norm) the domain content by fixing its latent to a mean prototype. Our simplified adaptation technique is not only clustering-free but also free from complex adversarial alignment. We achieve leading performance against the prior arts on the OCDA scene segmentation benchmarks.