DPStyler: Dynamic PromptStyler for Source-Free Domain Generalization

TL;DR

DPStyler is a novel approach for source-free domain generalization that uses dynamic text prompts to simulate diverse styles and extract domain-invariant features, improving model robustness across unseen domains.

Contribution

The paper introduces DPStyler, which employs style generation and removal modules along with ensemble techniques to enhance domain generalization without source data.

Findings

Outperforms state-of-the-art methods on benchmark datasets.

Effectively simulates diverse styles using text prompts.

Reduces style-induced variations in features.

Abstract

Source-Free Domain Generalization (SFDG) aims to develop a model that works for unseen target domains without relying on any source domain. Research in SFDG primarily bulids upon the existing knowledge of large-scale vision-language models and utilizes the pre-trained model's joint vision-language space to simulate style transfer across domains, thus eliminating the dependency on source domain images. However, how to efficiently simulate rich and diverse styles using text prompts, and how to extract domain-invariant information useful for classification from features that contain both semantic and style information after the encoder, are directions that merit improvement. In this paper, we introduce Dynamic PromptStyler (DPStyler), comprising Style Generation and Style Removal modules to address these issues. The Style Generation module refreshes all styles at every training epoch, while the Style Removal module eliminates variations in the encoder's output features caused by input styles. Moreover, since the Style Generation module, responsible for generating style word vectors using random sampling or style mixing, makes the model sensitive to input text prompts, we introduce a model ensemble method to mitigate this sensitivity. Extensive experiments demonstrate that our framework outperforms state-of-the-art methods on benchmark datasets.