Barycentric-alignment and reconstruction loss minimization for domain generalization

TL;DR

This paper introduces a new theoretical upper bound for domain generalization that incorporates distribution alignment and reconstruction, leading to a novel algorithm that outperforms existing methods.

Contribution

We derive a fully optimizable upper bound for unseen domain risk using transport inequalities, and propose the Wasserstein Barycenter Auto-Encoder (WBAE) for improved domain generalization.

Findings

WBAE outperforms state-of-the-art DG methods on multiple datasets.

The new upper bound effectively bridges theory and practice in DG.

Distribution alignment and reconstruction are key to improved generalization.

Abstract

This paper advances the theory and practice of Domain Generalization (DG) in machine learning. We consider the typical DG setting where the hypothesis is composed of a representation mapping followed by a labeling function. Within this setting, the majority of popular DG methods aim to jointly learn the representation and the labeling functions by minimizing a well-known upper bound for the classification risk in the unseen domain. In practice, however, methods based on this theoretical upper bound ignore a term that cannot be directly optimized due to its dual dependence on both the representation mapping and the unknown optimal labeling function in the unseen domain. To bridge this gap between theory and practice, we introduce a new upper bound that is free of terms having such dual dependence, resulting in a fully optimizable risk upper bound for the unseen domain. Our derivation leverages classical and recent transport inequalities that link optimal transport metrics with information-theoretic measures. Compared to previous bounds, our bound introduces two new terms: (i) the Wasserstein-2 barycenter term that aligns distributions between domains, and (ii) the reconstruction loss term that assesses the quality of representation in reconstructing the original data. Based on this new upper bound, we propose a novel DG algorithm named Wasserstein Barycenter Auto-Encoder (WBAE) that simultaneously minimizes the classification loss, the barycenter loss, and the reconstruction loss. Numerical results demonstrate that the proposed method outperforms current state-of-the-art DG algorithms on several datasets.