All about Structure: Adapting Structural Information across Domains for Boosting Semantic Segmentation

TL;DR

This paper introduces DISE, a framework for unsupervised domain adaptation in semantic segmentation that leverages shared structural information across domains to improve performance without requiring annotations in the target domain.

Contribution

The paper proposes a novel domain-invariant structure extraction method that disentangles images into shared structural and domain-specific texture features for better domain adaptation.

Findings

DISE outperforms state-of-the-art methods in unsupervised domain adaptation for semantic segmentation.

Structural content is highly effective for transferring knowledge across domains.

The approach improves segmentation accuracy on real-world images using synthetic training data.

Abstract

In this paper we tackle the problem of unsupervised domain adaptation for the task of semantic segmentation, where we attempt to transfer the knowledge learned upon synthetic datasets with ground-truth labels to real-world images without any annotation. With the hypothesis that the structural content of images is the most informative and decisive factor to semantic segmentation and can be readily shared across domains, we propose a Domain Invariant Structure Extraction (DISE) framework to disentangle images into domain-invariant structure and domain-specific texture representations, which can further realize image-translation across domains and enable label transfer to improve segmentation performance. Extensive experiments verify the effectiveness of our proposed DISE model and demonstrate its superiority over several state-of-the-art approaches.