DSP: Dual Soft-Paste for Unsupervised Domain Adaptive Semantic Segmentation

TL;DR

The paper introduces DSP, a novel dual soft-paste method that enhances unsupervised domain adaptation for semantic segmentation by improving feature alignment and convergence speed.

Contribution

DSP employs a class-wise soft-paste strategy and combines output and feature-level alignment within a mean teacher framework for improved domain adaptation.

Findings

DSP outperforms state-of-the-art methods on benchmark datasets.

Faster convergence and higher accuracy achieved with DSP.

Effective class-wise sampling improves feature alignment.

Abstract

Unsupervised domain adaptation (UDA) for semantic segmentation aims to adapt a segmentation model trained on the labeled source domain to the unlabeled target domain. Existing methods try to learn domain invariant features while suffering from large domain gaps that make it difficult to correctly align discrepant features, especially in the initial training phase. To address this issue, we propose a novel Dual Soft-Paste (DSP) method in this paper. Specifically, DSP selects some classes from a source domain image using a long-tail class first sampling strategy and softly pastes the corresponding image patch on both the source and target training images with a fusion weight. Technically, we adopt the mean teacher framework for domain adaptation, where the pasted source and target images go through the student network while the original target image goes through the teacher network. Output-level alignment is carried out by aligning the probability maps of the target fused image from both networks using a weighted cross-entropy loss. In addition, feature-level alignment is carried out by aligning the feature maps of the source and target images from student network using a weighted maximum mean discrepancy loss. DSP facilitates the model learning domain-invariant features from the intermediate domains, leading to faster convergence and better performance. Experiments on two challenging benchmarks demonstrate the superiority of DSP over state-of-the-art methods. Code is available at \url{https://github.com/GaoLii/DSP}.