EUDA: An Efficient Unsupervised Domain Adaptation via Self-Supervised Vision Transformer

TL;DR

EUDA introduces an efficient unsupervised domain adaptation framework using a self-supervised vision transformer and a novel domain alignment loss, achieving comparable performance with significantly fewer trainable parameters.

Contribution

The paper proposes EUDA, a resource-efficient domain adaptation method employing self-supervised ViT and a combined loss, reducing trainable parameters while maintaining performance.

Findings

Achieves comparable domain adaptation performance with 42%-99.7% fewer trainable parameters.

Utilizes DINOv2 ViT as feature extractor with a simplified bottleneck.

Demonstrates effectiveness in resource-limited environments.

Abstract

Unsupervised domain adaptation (UDA) aims to mitigate the domain shift issue, where the distribution of training (source) data differs from that of testing (target) data. Many models have been developed to tackle this problem, and recently vision transformers (ViTs) have shown promising results. However, the complexity and large number of trainable parameters of ViTs restrict their deployment in practical applications. This underscores the need for an efficient model that not only reduces trainable parameters but also allows for adjustable complexity based on specific needs while delivering comparable performance. To achieve this, in this paper we introduce an Efficient Unsupervised Domain Adaptation (EUDA) framework. EUDA employs the DINOv2, which is a self-supervised ViT, as a feature extractor followed by a simplified bottleneck of fully connected layers to refine features for enhanced domain adaptation. Additionally, EUDA employs the synergistic domain alignment loss (SDAL), which integrates cross-entropy (CE) and maximum mean discrepancy (MMD) losses, to balance adaptation by minimizing classification errors in the source domain while aligning the source and target domain distributions. The experimental results indicate the effectiveness of EUDA in producing comparable results as compared with other state-of-the-art methods in domain adaptation with significantly fewer trainable parameters, between 42% to 99.7% fewer. This showcases the ability to train the model in a resource-limited environment. The code of the model is available at: https://github.com/A-Abedi/EUDA.