Diffusion-based Image Translation with Label Guidance for Domain Adaptive Semantic Segmentation

TL;DR

This paper introduces a diffusion-based image translation method guided by source labels to improve domain adaptive semantic segmentation, effectively preserving local details across large domain gaps.

Contribution

It proposes a novel Semantic Gradient Guidance technique within a diffusion framework and a Progressive Translation Learning strategy for robust cross-domain translation.

Findings

Outperforms state-of-the-art methods in domain adaptive segmentation

Effectively preserves local semantic details during translation

Handles large domain gaps reliably

Abstract

Translating images from a source domain to a target domain for learning target models is one of the most common strategies in domain adaptive semantic segmentation (DASS). However, existing methods still struggle to preserve semantically-consistent local details between the original and translated images. In this work, we present an innovative approach that addresses this challenge by using source-domain labels as explicit guidance during image translation. Concretely, we formulate cross-domain image translation as a denoising diffusion process and utilize a novel Semantic Gradient Guidance (SGG) method to constrain the translation process, conditioning it on the pixel-wise source labels. Additionally, a Progressive Translation Learning (PTL) strategy is devised to enable the SGG method to work reliably across domains with large gaps. Extensive experiments demonstrate the superiority of our approach over state-of-the-art methods.