Do-PFN: In-Context Learning for Causal Effect Estimation

TL;DR

This paper introduces Do-PFN, a method that uses in-context learning with pre-trained neural networks to estimate causal effects accurately from observational data without needing the causal graph or interventional data.

Contribution

It demonstrates that PFNs can be adapted for causal effect estimation, enabling accurate predictions across diverse causal structures without prior causal knowledge.

Findings

Accurately estimates causal effects without causal graph knowledge

Performs well across various synthetic causal structures

Shows robustness and scalability in experiments

Abstract

Estimation of causal effects is critical to a range of scientific disciplines. Existing methods for this task either require interventional data, knowledge about the ground truth causal graph, or rely on assumptions such as unconfoundedness, restricting their applicability in real-world settings. In the domain of tabular machine learning, Prior-data fitted networks (PFNs) have achieved state-of-the-art predictive performance, having been pre-trained on synthetic data to solve tabular prediction problems via in-context learning. To assess whether this can be transferred to the harder problem of causal effect estimation, we pre-train PFNs on synthetic data drawn from a wide variety of causal structures, including interventions, to predict interventional outcomes given observational data. Through extensive experiments on synthetic case studies, we show that our approach allows for the accurate estimation of causal effects without knowledge of the underlying causal graph. We also perform ablation studies that elucidate Do-PFN's scalability and robustness across datasets with a variety of causal characteristics.