Pulling Target to Source: A New Perspective on Domain Adaptive Semantic Segmentation

TL;DR

This paper introduces T2S-DA, a novel domain adaptation method for semantic segmentation that pulls target features towards source features per category, improving discriminability and generalization across domains.

Contribution

The work proposes a new perspective of pulling target features to source features for domain adaptation, along with a dynamic re-weighting strategy for class imbalance.

Findings

T2S-DA outperforms state-of-the-art methods in domain adaptive segmentation.

The method enhances feature discriminability and generalization.

It is effective for domain generalization tasks.

Abstract

Domain adaptive semantic segmentation aims to transfer knowledge from a labeled source domain to an unlabeled target domain. However, existing methods primarily focus on directly learning qualified target features, making it challenging to guarantee their discrimination in the absence of target labels. This work provides a new perspective. We observe that the features learned with source data manage to keep categorically discriminative during training, thereby enabling us to implicitly learn adequate target representations by simply \textbf{pulling target features close to source features for each category}. To this end, we propose T2S-DA, which we interpret as a form of pulling Target to Source for Domain Adaptation, encouraging the model in learning similar cross-domain features. Also, considering the pixel categories are heavily imbalanced for segmentation datasets, we come up with a dynamic re-weighting strategy to help the model concentrate on those underperforming classes. Extensive experiments confirm that T2S-DA learns a more discriminative and generalizable representation, significantly surpassing the state-of-the-art. We further show that our method is quite qualified for the domain generalization task, verifying its domain-invariant property.