Conformal Counterfactual Inference under Hidden Confounding

TL;DR

This paper introduces a novel conformal prediction method for counterfactual outcomes that remains valid under hidden confounding and requires less restrictive assumptions, improving confidence interval accuracy in causal inference.

Contribution

The paper proposes wTCP-DR, a transductive weighted conformal prediction approach that provides valid confidence intervals for counterfactuals even with hidden confounding and limited interventional data.

Findings

Outperforms state-of-the-art baselines in coverage and efficiency.

Provides theoretical guarantees under weaker assumptions.

Validated on synthetic and real-world datasets, including recommendation systems.

Abstract

Personalized decision making requires the knowledge of potential outcomes under different treatments, and confidence intervals about the potential outcomes further enrich this decision-making process and improve its reliability in high-stakes scenarios. Predicting potential outcomes along with its uncertainty in a counterfactual world poses the foundamental challenge in causal inference. Existing methods that construct confidence intervals for counterfactuals either rely on the assumption of strong ignorability, or need access to un-identifiable lower and upper bounds that characterize the difference between observational and interventional distributions. To overcome these limitations, we first propose a novel approach wTCP-DR based on transductive weighted conformal prediction, which provides confidence intervals for counterfactual outcomes with marginal converage guarantees, even under hidden confounding. With less restrictive assumptions, our approach requires access to a fraction of interventional data (from randomized controlled trials) to account for the covariate shift from observational distributoin to interventional distribution. Theoretical results explicitly demonstrate the conditions under which our algorithm is strictly advantageous to the naive method that only uses interventional data. After ensuring valid intervals on counterfactuals, it is straightforward to construct intervals for individual treatment effects (ITEs). We demonstrate our method across synthetic and real-world data, including recommendation systems, to verify the superiority of our methods compared against state-of-the-art baselines in terms of both coverage and efficiency