Joint covariate-alignment and concept-alignment: a framework for domain generalization

TL;DR

This paper introduces a new domain generalization framework that jointly minimizes covariate and concept shifts using distributional alignment and invariant risk minimization, improving performance on unseen domains.

Contribution

It proposes a novel joint covariate- and concept-alignment framework combining MMD, CORAL, and IRM for better domain generalization.

Findings

Performs as well as or better than state-of-the-art methods.

Effective in reducing covariate and concept shifts.

Improves unseen domain risk on multiple datasets.

Abstract

In this paper, we propose a novel domain generalization (DG) framework based on a new upper bound to the risk on the unseen domain. Particularly, our framework proposes to jointly minimize both the covariate-shift as well as the concept-shift between the seen domains for a better performance on the unseen domain. While the proposed approach can be implemented via an arbitrary combination of covariate-alignment and concept-alignment modules, in this work we use well-established approaches for distributional alignment namely, Maximum Mean Discrepancy (MMD) and covariance Alignment (CORAL), and use an Invariant Risk Minimization (IRM)-based approach for concept alignment. Our numerical results show that the proposed methods perform as well as or better than the state-of-the-art for domain generalization on several data sets.