Learning Texture Invariant Representation for Domain Adaptation of Semantic Segmentation

TL;DR

This paper proposes a texture adaptation method for semantic segmentation that enhances model generalization from synthetic to real images by diversifying textures and fine-tuning with self-training, achieving state-of-the-art results.

Contribution

It introduces a novel texture adaptation approach using style transfer and self-training to improve domain adaptation in semantic segmentation.

Findings

Achieves state-of-the-art performance on benchmark datasets.

Texture diversification prevents overfitting to synthetic textures.

Self-training effectively adapts models to real target textures.

Abstract

Since annotating pixel-level labels for semantic segmentation is laborious, leveraging synthetic data is an attractive solution. However, due to the domain gap between synthetic domain and real domain, it is challenging for a model trained with synthetic data to generalize to real data. In this paper, considering the fundamental difference between the two domains as the texture, we propose a method to adapt to the texture of the target domain. First, we diversity the texture of synthetic images using a style transfer algorithm. The various textures of generated images prevent a segmentation model from overfitting to one specific (synthetic) texture. Then, we fine-tune the model with self-training to get direct supervision of the target texture. Our results achieve state-of-the-art performance and we analyze the properties of the model trained on the stylized dataset with extensive experiments.