Domain Agnostic Conditional Invariant Predictions for Domain Generalization

TL;DR

This paper introduces a domain-agnostic approach for domain generalization that learns invariant features without relying on domain labels, using a new theoretical framework and an algorithm combining Bayesian inference and a novel penalty.

Contribution

The paper proposes DRM theory and an algorithm that captures invariant features without domain labels, advancing domain generalization methods.

Findings

Effective in learning invariant features across domains

Outperforms existing methods on multiple datasets

Validates DRM theory through empirical results

Abstract

Domain generalization aims to develop a model that can perform well on unseen target domains by learning from multiple source domains. However, recent-proposed domain generalization models usually rely on domain labels, which may not be available in many real-world scenarios. To address this challenge, we propose a Discriminant Risk Minimization (DRM) theory and the corresponding algorithm to capture the invariant features without domain labels. In DRM theory, we prove that reducing the discrepancy of prediction distribution between overall source domain and any subset of it can contribute to obtaining invariant features. To apply the DRM theory, we develop an algorithm which is composed of Bayesian inference and a new penalty termed as Categorical Discriminant Risk (CDR). In Bayesian inference, we transform the output of the model into a probability distribution to align with our theoretical assumptions. We adopt sliding update approach to approximate the overall prediction distribution of the model, which enables us to obtain CDR penalty. We also indicate the effectiveness of these components in finding invariant features. We evaluate our algorithm against various domain generalization methods on multiple real-world datasets, providing empirical support for our theory.