Vision transformers in domain adaptation and domain generalization: a study of robustness

TL;DR

This paper reviews how vision transformers are applied to improve robustness and generalization in domain adaptation and domain generalization tasks, highlighting their potential in handling distribution shifts in computer vision.

Contribution

It provides a comprehensive categorization and analysis of vision transformer-based methods across various strategies for domain adaptation and generalization, with detailed summaries.

Findings

Vision transformers show versatility in managing distribution shifts.

Categorization of methods enhances understanding of current approaches.

Summarized strategies serve as a guide for future research.

Abstract

Deep learning models are often evaluated in scenarios where the data distribution is different from those used in the training and validation phases. The discrepancy presents a challenge for accurately predicting the performance of models once deployed on the target distribution. Domain adaptation and generalization are widely recognized as effective strategies for addressing such shifts, thereby ensuring reliable performance. The recent promising results in applying vision transformers in computer vision tasks, coupled with advancements in self-attention mechanisms, have demonstrated their significant potential for robustness and generalization in handling distribution shifts. Motivated by the increased interest from the research community, our paper investigates the deployment of vision transformers in domain adaptation and domain generalization scenarios. For domain adaptation methods, we categorize research into feature-level, instance-level, model-level adaptations, and hybrid approaches, along with other categorizations with respect to diverse strategies for enhancing domain adaptation. Similarly, for domain generalization, we categorize research into multi-domain learning, meta-learning, regularization techniques, and data augmentation strategies. We further classify diverse strategies in research, underscoring the various approaches researchers have taken to address distribution shifts by integrating vision transformers. The inclusion of comprehensive tables summarizing these categories is a distinct feature of our work, offering valuable insights for researchers. These findings highlight the versatility of vision transformers in managing distribution shifts, crucial for real-world applications, especially in critical safety and decision-making scenarios.