Prediction-Powered Inference with Inverse Probability Weighting

TL;DR

This paper introduces a new interpretation of prediction-powered inference (PPI) that integrates survey sampling techniques, allowing for valid inference with estimated inclusion probabilities and partially labeled data.

Contribution

It provides a direct design-based interpretation of PPI, connecting it with survey sampling methods like Horvitz--Thompson corrections, and demonstrates its effectiveness with estimated propensities.

Findings

IPW-adjusted PPI with estimated propensities maintains nominal coverage.

Performance closely matches the known-probability case in simulations.

Retains variance reduction benefits of PPI.

Abstract

Prediction-powered inference (PPI) is a recent framework for valid statistical inference with partially labeled data, combining model-based predictions on a large unlabeled set with bias correction from a smaller labeled subset. Building on existing PPI results under covariate shift, we show that PPI rectification admits a direct design-based interpretation, and that informative labeling can be handled naturally by Horvitz--Thompson and H\'ajek-style corrections. This connection unites design-based survey sampling ideas with modern prediction-assisted inference, yielding estimators that remain valid when labeling probabilities vary across units. We consider the common setting where the inclusion probabilities are not known but estimated from a correctly specified model. In simulations, the performance of IPW-adjusted PPI with estimated propensities closely matches the known-probability case, retaining both nominal coverage and the variance-reduction benefits of PPI.