Anytime-valid, Bayes-assisted, Prediction-Powered Inference

TL;DR

This paper extends prediction-powered inference to sequential data, creating confidence sequences that adapt over time, incorporate prior knowledge, and maintain validity, thereby improving statistical efficiency in dynamic settings.

Contribution

It introduces a sequential prediction-powered confidence sequence framework using Ville's inequality and mixture methods, enhancing efficiency and validity over time.

Findings

Effective in real and synthetic examples

Maintains fixed-time validity

Increases efficiency with prior knowledge

Abstract

Given a large pool of unlabelled data and a smaller amount of labels, prediction-powered inference (PPI) leverages machine learning predictions to increase the statistical efficiency of confidence interval procedures based solely on labelled data, while preserving fixed-time validity. In this paper, we extend the PPI framework to the sequential setting, where labelled and unlabelled datasets grow over time. Exploiting Ville's inequality and the method of mixtures, we propose prediction-powered confidence sequence procedures that are asymptotically valid uniformly over time and naturally accommodate prior knowledge on the quality of the predictions to further boost efficiency. We carefully illustrate the design choices behind our method and demonstrate its effectiveness in real and synthetic examples.