Instrumental Variable-Driven Domain Generalization with Unobserved Confounders

TL;DR

This paper introduces IV-DG, a novel domain generalization approach leveraging instrumental variables to address unobserved confounders, improving invariant relationship learning across unseen domains with theoretical and empirical validation.

Contribution

The paper proposes a causal, instrumental variable-based method for domain generalization that effectively removes bias from unobserved confounders, a novel approach in this context.

Findings

Achieves state-of-the-art results on real-world datasets.

Accurately captures invariant relationships across domains.

Theoretically validated through analysis and simulations.

Abstract

Domain generalization (DG) aims to learn from multiple source domains a model that can generalize well on unseen target domains. Existing DG methods mainly learn the representations with invariant marginal distribution of the input features, however, the invariance of the conditional distribution of the labels given the input features is more essential for unknown domain prediction. Meanwhile, the existing of unobserved confounders which affect the input features and labels simultaneously cause spurious correlation and hinder the learning of the invariant relationship contained in the conditional distribution. Interestingly, with a causal view on the data generating process, we find that the input features of one domain are valid instrumental variables for other domains. Inspired by this finding, we propose an instrumental variable-driven DG method (IV-DG) by removing the bias of the unobserved confounders with two-stage learning. In the first stage, it learns the conditional distribution of the input features of one domain given input features of another domain. In the second stage, it estimates the relationship by predicting labels with the learned conditional distribution. Theoretical analyses and simulation experiments show that it accurately captures the invariant relationship. Extensive experiments on real-world datasets demonstrate that IV-DG method yields state-of-the-art results.