Causality Pursuit from Heterogeneous Environments via Neural Adversarial Invariance Learning

TL;DR

This paper proposes a neural adversarial invariance learning framework, FAIR, for discovering causal variables across heterogeneous environments, addressing challenges of nonparametric invariance and endogenous effects.

Contribution

It introduces a novel minimax neural network approach, FAIR-NN, for causality pursuit that can identify invariant and quasi-causal variables under minimal conditions.

Findings

FAIR-NN effectively finds invariant variables in simulated data.

The method aligns with true causal mechanisms under sufficient heterogeneity.

Demonstrated success on real-world datasets.

Abstract

Pursuing causality from data is a fundamental problem in scientific discovery, treatment intervention, and transfer learning. This paper introduces a novel algorithmic method for addressing nonparametric invariance and causality learning in regression models across multiple environments, where the joint distribution of response variables and covariates varies, but the conditional expectations of outcome given an unknown set of quasi-causal variables are invariant. The challenge of finding such an unknown set of quasi-causal or invariant variables is compounded by the presence of endogenous variables that have heterogeneous effects across different environments. The proposed Focused Adversarial Invariant Regularization (FAIR) framework utilizes an innovative minimax optimization approach that drives regression models toward prediction-invariant solutions through adversarial testing. Leveraging the representation power of neural networks, FAIR neural networks (FAIR-NN) are introduced for causality pursuit. It is shown that FAIR-NN can find the invariant variables and quasi-causal variables under a minimal identification condition and that the resulting procedure is adaptive to low-dimensional composition structures in a non-asymptotic analysis. Under a structural causal model, variables identified by FAIR-NN represent pragmatic causality and provably align with exact causal mechanisms under conditions of sufficient heterogeneity. Computationally, FAIR-NN employs a novel Gumbel approximation with decreased temperature and a stochastic gradient descent ascent algorithm. The procedures are demonstrated using simulated and real-data examples.