Prediction-Powered Causal Inferences

TL;DR

This paper introduces a new method for causal inference in scientific experiments using machine learning predictions, ensuring valid treatment effect estimates across different experiments without requiring extensive annotations.

Contribution

The paper proposes a novel training objective called Deconfounded Empirical Risk Minimization that guarantees valid causal inferences across experiments, even with complex data and minimal annotations.

Findings

Validated on synthetic and real data, solving previously impossible instances.

Achieved valid causal inference on complex, unannotated scientific data.

Demonstrated effectiveness of the method in transferring validity across experiments.

Abstract

In many scientific experiments, the data annotating cost constraints the pace for testing novel hypotheses. Yet, modern machine learning pipelines offer a promising solution, provided their predictions yield correct conclusions. We focus on Prediction-Powered Causal Inferences (PPCI), i.e., estimating the treatment effect in an unlabeled target experiment, relying on training data with the same outcome annotated but potentially different treatment or effect modifiers. We first show that conditional calibration guarantees valid PPCI at population level. Then, we introduce a sufficient representation constraint transferring validity across experiments, which we propose to enforce in practice in Deconfounded Empirical Risk Minimization, our new model-agnostic training objective. We validate our method on synthetic and real-world scientific data, solving impossible problem instances for Empirical Risk Minimization even with standard invariance constraints. In particular, for the first time, we achieve valid causal inference on a scientific experiment with complex recording and no human annotations, fine-tuning a foundational model on our similar annotated experiment.