Dual-Foundation Models for Unsupervised Domain Adaptation

TL;DR

This paper introduces a dual-foundation model approach for unsupervised domain adaptation in semantic segmentation, utilizing SAM and DINOv3 to improve learning from unlabeled real images.

Contribution

It proposes a novel framework combining SAM and DINOv3 to address pseudo-label reliance and prototype bias in UDA for semantic segmentation.

Findings

Achieves +1.3% mIoU on GTA-to-Cityscapes

Achieves +1.4% mIoU on SYNTHIA-to-Cityscapes

Outperforms strong UDA baselines

Abstract

Semantic segmentation provides pixel-level scene understanding essential for autonomous driving and fine-grained perception tasks. However, training segmentation models requires costly, labor-intensive annotations on real-world datasets. Unsupervised Domain Adaptation (UDA) addresses this by training models on labeled synthetic data and adapting them to unlabeled real images. While conceptually simple, adaptation is challenging due to the domain gap, i.e., differences in visual appearance and scene structure between synthetic and real data. Prior approaches bridge this gap through pixel-level mixing or feature-level contrastive learning. Yet, these techniques suffer from two major limitations: (1) reliance on high-confidence pseudo-labels restricts learning to a subset of the target domain, and (2) prototype-based contrastive methods initialize class prototypes from source-trained models, yielding biased and unstable anchors during adaptation. To address these issues, we propose a dual-foundation UDA framework that leverages two complementary foundation models. First, we employ the Segment Anything Model (SAM) with superpixel-guided prompting to enable learning from a broader range of target pixels beyond high-confidence predictions. Second, we incorporate DINOv3 to construct stable, domain-invariant class prototypes through its robust representation learning. Our method achieves consistent improvements of +1.3% and +1.4% mIoU over strong UDA baselines on GTA-to-Cityscapes and SYNTHIA-to-Cityscapes, respectively.