Tackling Dimensional Collapse toward Comprehensive Universal Domain Adaptation

TL;DR

This paper identifies dimensional collapse as a key issue in universal domain adaptation and proposes self-supervised de-collapse techniques to improve representation structure, achieving state-of-the-art results across diverse scenarios.

Contribution

It introduces a novel de-collapse method using self-supervised learning to enhance partial domain matching in universal domain adaptation.

Findings

Self-supervised de-collapse improves target representations.

The method achieves state-of-the-art results on multiple UniDA benchmarks.

Addresses extreme cases with many absent source classes.

Abstract

Universal Domain Adaptation (UniDA) addresses unsupervised domain adaptation where target classes may differ arbitrarily from source ones, except for a shared subset. A widely used approach, partial domain matching (PDM), aligns only shared classes but struggles in extreme cases where many source classes are absent in the target domain, underperforming the most naive baseline that trains on only source data. In this work, we identify that the failure of PDM for extreme UniDA stems from dimensional collapse (DC) in target representations. To address target DC, we propose to use the de-collapse techniques in self-supervised learning on the unlabeled target data to preserve the intrinsic structure of the learned representations. Our experimental results confirm that SSL consistently advances PDM and delivers new state-of-the-art results across a broader benchmark of UniDA scenarios with different portions of shared classes, representing a crucial step toward truly comprehensive UniDA. Project page: https://dc-unida.github.io/