I2F: A Unified Image-to-Feature Approach for Domain Adaptive Semantic Segmentation

TL;DR

This paper introduces a unified approach for unsupervised domain adaptation in semantic segmentation, combining image-level and feature-level alignment techniques to improve performance across different domains.

Contribution

It proposes a novel pipeline that unifies image-level and feature-level adaptation methods, including photometric and texture alignment, manifold alignment, and category regularization.

Findings

Outperforms previous methods significantly.

Achieves 58.2% mIoU on GTA5 to Cityscapes.

Surpasses SOTA by 8% in the benchmark task.

Abstract

Unsupervised domain adaptation (UDA) for semantic segmentation is a promising task freeing people from heavy annotation work. However, domain discrepancies in low-level image statistics and high-level contexts compromise the segmentation performance over the target domain. A key idea to tackle this problem is to perform both image-level and feature-level adaptation jointly. Unfortunately, there is a lack of such unified approaches for UDA tasks in the existing literature. This paper proposes a novel UDA pipeline for semantic segmentation that unifies image-level and feature-level adaptation. Concretely, for image-level domain shifts, we propose a global photometric alignment module and a global texture alignment module that align images in the source and target domains in terms of image-level properties. For feature-level domain shifts, we perform global manifold alignment by projecting pixel features from both domains onto the feature manifold of the source domain; and we further regularize category centers in the source domain through a category-oriented triplet loss and perform target domain consistency regularization over augmented target domain images. Experimental results demonstrate that our pipeline significantly outperforms previous methods. In the commonly tested GTA5$\rightarrow$Cityscapes task, our proposed method using Deeplab V3+ as the backbone surpasses previous SOTA by 8%, achieving 58.2% in mIoU.