Affinity Space Adaptation for Semantic Segmentation Across Domains

TL;DR

This paper introduces a novel unsupervised domain adaptation method for semantic segmentation that exploits invariant affinity relationships between adjacent pixels across domains, improving cross-domain performance.

Contribution

It proposes affinity space adaptation strategies, including affinity space cleaning and adversarial alignment, to better transfer semantic segmentation models across different domains.

Findings

Achieves superior performance on multiple benchmarks.

Outperforms several state-of-the-art methods.

Effectively leverages pixel affinity invariance across domains.

Abstract

Semantic segmentation with dense pixel-wise annotation has achieved excellent performance thanks to deep learning. However, the generalization of semantic segmentation in the wild remains challenging. In this paper, we address the problem of unsupervised domain adaptation (UDA) in semantic segmentation. Motivated by the fact that source and target domain have invariant semantic structures, we propose to exploit such invariance across domains by leveraging co-occurring patterns between pairwise pixels in the output of structured semantic segmentation. This is different from most existing approaches that attempt to adapt domains based on individual pixel-wise information in image, feature, or output level. Specifically, we perform domain adaptation on the affinity relationship between adjacent pixels termed affinity space of source and target domain. To this end, we develop two affinity space adaptation strategies: affinity space cleaning and adversarial affinity space alignment. Extensive experiments demonstrate that the proposed method achieves superior performance against some state-of-the-art methods on several challenging benchmarks for semantic segmentation across domains. The code is available at https://github.com/idealwei/ASANet.