Semantic Bridging Domains: Pseudo-Source as Test-Time Connector

TL;DR

This paper introduces a Stepwise Semantic Alignment method that uses pseudo-source domains as semantic bridges to improve model adaptation across distribution shifts, achieving significant performance gains.

Contribution

The paper proposes a novel SSA approach that leverages universal semantics and introduces HFA and CACL modules for better domain adaptation without source labels.

Findings

Achieved 5.2% performance boost on GTA2Cityscapes.

Effectively aligns target domain using corrected pseudo-source semantics.

Enhances semantic quality with HFA and CACL modules.

Abstract

Distribution shifts between training and testing data are a critical bottleneck limiting the practical utility of models, especially in real-world test-time scenarios. To adapt models when the source domain is unknown and the target domain is unlabeled, previous works constructed pseudo-source domains via data generation and translation, then aligned the target domain with them. However, significant discrepancies exist between the pseudo-source and the original source domain, leading to potential divergence when correcting the target directly. From this perspective, we propose a Stepwise Semantic Alignment (SSA) method, viewing the pseudo-source as a semantic bridge connecting the source and target, rather than a direct substitute for the source. Specifically, we leverage easily accessible universal semantics to rectify the semantic features of the pseudo-source, and then align the target domain using the corrected pseudo-source semantics. Additionally, we introduce a Hierarchical Feature Aggregation (HFA) module and a Confidence-Aware Complementary Learning (CACL) strategy to enhance the semantic quality of the SSA process in the absence of source and ground truth of target domains. We evaluated our approach on tasks like semantic segmentation and image classification, achieving a 5.2% performance boost on GTA2Cityscapes over the state-of-the-art.