Fundamental Computational Limits in Pursuing Invariant Causal Prediction and Invariance-Guided Regularization

TL;DR

This paper demonstrates the intrinsic computational hardness of invariant causal prediction, proving NP-hardness even for linear cases, and proposes a robust estimation method that balances computational efficiency and statistical accuracy under certain conditions.

Contribution

It establishes the NP-hardness of testing invariant causal solutions and introduces a distributionally robust estimator that interpolates between predictive and causal solutions.

Findings

NP-hardness of invariant causal prediction testing even for linear models

Proposed estimator achieves computational and statistical efficiency under certain conditions

Empirical results support the effectiveness of the robust estimation method

Abstract

Pursuing invariant prediction from heterogeneous environments opens the door to learning causality in a purely data-driven way and has several applications in causal discovery and robust transfer learning. However, existing methods such as ICP [Peters et al., 2016] and EILLS [Fan et al., 2024] that can attain sample-efficient estimation are based on exponential time algorithms. In this paper, we show that such a problem is intrinsically hard in computation: the decision problem, testing whether a non-trivial prediction-invariant solution exists across two environments, is NP-hard even for the linear causal relationship. In the world where P$\neq$NP, our results imply that the estimation error rate can be arbitrarily slow using any computationally efficient algorithm. This suggests that pursuing causality is fundamentally harder than detecting associations when no prior assumption is pre-offered. Given there is almost no hope of computational improvement under the worst case, this paper proposes a method capable of attaining both computationally and statistically efficient estimation under additional conditions. Furthermore, our estimator is a distributionally robust estimator with an ellipse-shaped uncertain set where more uncertainty is placed on spurious directions than invariant directions, resulting in a smooth interpolation between the most predictive solution and the causal solution by varying the invariance hyper-parameter. Non-asymptotic results and empirical applications support the claim.