Revisiting Spurious Correlation in Domain Generalization

TL;DR

This paper revisits the problem of spurious correlations in domain generalization, proposing a new SCM for representation learning and a propensity score weighted estimator to improve out-of-distribution generalization.

Contribution

It introduces a novel SCM tailored for representation learning and a plug-and-play bias control method for better OOD generalization.

Findings

Effective on synthetic datasets

Improves OOD performance on real datasets

Validates the proposed approach through extensive experiments

Abstract

Without loss of generality, existing machine learning techniques may learn spurious correlation dependent on the domain, which exacerbates the generalization of models in out-of-distribution (OOD) scenarios. To address this issue, recent works build a structural causal model (SCM) to describe the causality within data generation process, thereby motivating methods to avoid the learning of spurious correlation by models. However, from the machine learning viewpoint, such a theoretical analysis omits the nuanced difference between the data generation process and representation learning process, resulting in that the causal analysis based on the former cannot well adapt to the latter. To this end, we explore to build a SCM for representation learning process and further conduct a thorough analysis of the mechanisms underlying spurious correlation. We underscore that adjusting erroneous covariates introduces bias, thus necessitating the correct selection of spurious correlation mechanisms based on practical application scenarios. In this regard, we substantiate the correctness of the proposed SCM and further propose to control confounding bias in OOD generalization by introducing a propensity score weighted estimator, which can be integrated into any existing OOD method as a plug-and-play module. The empirical results comprehensively demonstrate the effectiveness of our method on synthetic and large-scale real OOD datasets.