SemAlign: Language Guided Semi-supervised Domain Generalization

TL;DR

SemAlign introduces a novel semi-supervised domain generalization method that aligns model features with a vision-language model's semantic space, enhancing domain-invariance and data utilization, leading to state-of-the-art results.

Contribution

The paper proposes a new approach that aligns features with a vision-language model to improve domain generalization in semi-supervised settings.

Findings

Achieves state-of-the-art results on four benchmarks.

Effectively utilizes data with augmentation and regularization.

Outperforms existing SSDG baselines.

Abstract

Semi-supervised Domain Generalization (SSDG) addresses the challenge of generalizing to unseen target domains with limited labeled data. Existing SSDG methods highlight the importance of achieving high pseudo-labeling (PL) accuracy and preventing model overfitting as the main challenges in SSDG. In this light, we show that the SSDG literature's excessive focus on PL accuracy, without consideration for maximum data utilization during training, limits potential performance improvements. We propose a novel approach to the SSDG problem by aligning the intermediate features of our model with the semantically rich and generalized feature space of a Vision Language Model (VLM) in a way that promotes domain-invariance. The above approach is enhanced with effective image-level augmentation and output-level regularization strategies to improve data utilization and minimize overfitting. Extensive experimentation across four benchmarks against existing SSDG baselines suggests that our method achieves SOTA results both qualitatively and quantitatively. The code will be made publicly available.