Text-Driven Causal Representation Learning for Source-Free Domain Generalization

TL;DR

This paper introduces TDCRL, a novel causal inference-based method for source-free domain generalization that leverages text prompts and causal intervention to improve robustness across diverse visual domains.

Contribution

TDCRL is the first approach to incorporate causal inference into source-free domain generalization using text-driven representations.

Findings

Achieves state-of-the-art results on multiple benchmarks.

Effectively handles domain-specific confounders.

Demonstrates robustness and improved generalization.

Abstract

Deep learning often struggles when training and test data distributions differ. Traditional domain generalization (DG) tackles this by including data from multiple source domains, which is impractical due to expensive data collection and annotation. Recent vision-language models like CLIP enable source-free domain generalization (SFDG) by using text prompts to simulate visual representations, reducing data demands. However, existing SFDG methods struggle with domain-specific confounders, limiting their generalization capabilities. To address this issue, we propose TDCRL (\textbf{T}ext-\textbf{D}riven \textbf{C}ausal \textbf{R}epresentation \textbf{L}earning), the first method to integrate causal inference into the SFDG setting. TDCRL operates in two steps: first, it employs data augmentation to generate style word vectors, combining them with class information to generate text embeddings to simulate visual representations; second, it trains a causal intervention network with a confounder dictionary to extract domain-invariant features. Grounded in causal learning, our approach offers a clear and effective mechanism to achieve robust, domain-invariant features, ensuring robust generalization. Extensive experiments on PACS, VLCS, OfficeHome, and DomainNet show state-of-the-art performance, proving TDCRL effectiveness in SFDG.