Bi-Level Optimization for Single Domain Generalization

TL;DR

This paper introduces BiSDG, a bi-level optimization framework for single domain generalization that uses surrogate domains and domain prompts to improve unseen domain performance.

Contribution

The paper proposes a novel bi-level optimization approach with domain prompts and surrogate domains for effective single domain generalization.

Findings

BiSDG outperforms prior methods on multiple benchmarks.

The framework achieves state-of-the-art results in SDG.

Efficient gradient approximation enables practical training.

Abstract

Generalizing from a single labeled source domain to unseen target domains, without access to any target data during training, remains a fundamental challenge in robust machine learning. We address this underexplored setting, known as Single Domain Generalization (SDG), by proposing BiSDG, a bi-level optimization framework that explicitly decouples task learning from domain modeling. BiSDG simulates distribution shifts through surrogate domains constructed via label-preserving transformations of the source data. To capture domain-specific context, we propose a domain prompt encoder that generates lightweight modulation signals to produce augmenting features via feature-wise linear modulation. The learning process is formulated as a bi-level optimization problem: the inner objective optimizes task performance under fixed prompts, while the outer objective maximizes generalization across the surrogate domains by updating the domain prompt encoder. We further develop a practical gradient approximation scheme that enables efficient bi-level training without second-order derivatives. Extensive experiments on various SGD benchmarks demonstrate that BiSDG consistently outperforms prior methods, setting new state-of-the-art performance in the SDG setting.