Robust Domain Generalization under Divergent Marginal and Conditional Distributions

TL;DR

This paper introduces a unified framework for robust domain generalization that handles simultaneous divergence in both marginal and conditional distributions, improving generalization to unseen domains.

Contribution

It proposes a novel risk bound considering joint distribution shifts and a meta-learning approach to optimize for strong generalization across diverse domains.

Findings

Achieves state-of-the-art results on standard DG benchmarks.

Performs well in multi-domain long-tailed recognition scenarios.

Effectively handles complex distribution shifts in real-world data.

Abstract

Domain generalization (DG) aims to learn predictive models that can generalize to unseen domains. Most existing DG approaches focus on learning domain-invariant representations under the assumption of conditional distribution shift (i.e., primarily addressing changes in $P(X\mid Y)$ while assuming $P(Y)$ remains stable). However, real-world scenarios with multiple domains often involve compound distribution shifts where both the marginal label distribution $P(Y)$ and the conditional distribution $P(X\mid Y)$ vary simultaneously. To address this, we propose a unified framework for robust domain generalization under divergent marginal and conditional distributions. We derive a novel risk bound for unseen domains by explicitly decomposing the joint distribution into marginal and conditional components and characterizing risk gaps arising from both sources of divergence. To operationalize this bound, we design a meta-learning procedure that minimizes and validates the proposed risk bound across seen domains, ensuring strong generalization to unseen ones. Empirical evaluations demonstrate that our method achieves state-of-the-art performance not only on conventional DG benchmarks but also in challenging multi-domain long-tailed recognition settings where both marginal and conditional shifts are pronounced.