PromptTA: Prompt-driven Text Adapter for Source-free Domain Generalization

TL;DR

PromptTA introduces a prompt-driven text adapter that enhances source-free domain generalization by better capturing style features and employing resampling, leading to state-of-the-art results across multiple benchmarks.

Contribution

The paper proposes PromptTA, a novel text adapter that improves domain knowledge capture and generalization in source-free settings using style feature resampling.

Findings

Achieves state-of-the-art performance on four benchmark datasets.

Effectively captures diverse domain styles with resampling strategy.

Outperforms existing methods in source-free domain generalization.

Abstract

Source-free domain generalization (SFDG) tackles the challenge of adapting models to unseen target domains without access to source domain data. To deal with this challenging task, recent advances in SFDG have primarily focused on leveraging the text modality of vision-language models such as CLIP. These methods involve developing a transferable linear classifier based on diverse style features extracted from the text and learned prompts or deriving domain-unified text representations from domain banks. However, both style features and domain banks have limitations in capturing comprehensive domain knowledge. In this work, we propose Prompt-Driven Text Adapter (PromptTA) method, which is designed to better capture the distribution of style features and employ resampling to ensure thorough coverage of domain knowledge. To further leverage this rich domain information, we introduce a text adapter that learns from these style features for efficient domain information storage. Extensive experiments conducted on four benchmark datasets demonstrate that PromptTA achieves state-of-the-art performance. The code is available at https://github.com/zhanghr2001/PromptTA.