Direct-Effect Risk Minimization for Domain Generalization

TL;DR

This paper introduces a causal inference-based method to improve out-of-distribution generalization by minimizing indirect effects, effectively handling correlation shifts across domains.

Contribution

The paper proposes a novel two-stage algorithm that isolates and removes indirect effects to enhance domain generalization, compatible with existing methods.

Findings

Improved performance on 5 correlation-shifted datasets

Enhanced generalization on DomainBed benchmark

Effective removal of indirect effects

Abstract

We study the problem of out-of-distribution (o.o.d.) generalization where spurious correlations of attributes vary across training and test domains. This is known as the problem of correlation shift and has posed concerns on the reliability of machine learning. In this work, we introduce the concepts of direct and indirect effects from causal inference to the domain generalization problem. We argue that models that learn direct effects minimize the worst-case risk across correlation-shifted domains. To eliminate the indirect effects, our algorithm consists of two stages: in the first stage, we learn an indirect-effect representation by minimizing the prediction error of domain labels using the representation and the class labels; in the second stage, we remove the indirect effects learned in the first stage by matching each data with another data of similar indirect-effect representation but of different class labels in the training and validation phase. Our approach is shown to be compatible with existing methods and improve the generalization performance of them on correlation-shifted datasets. Experiments on 5 correlation-shifted datasets and the DomainBed benchmark verify the effectiveness of our approach.