Single Trajectory Conformal Prediction

TL;DR

This paper analyzes the performance of risk-controlling prediction sets in conformal prediction when applied to temporally correlated data from unknown dynamical systems, providing guarantees under stationarity and insights into deviations.

Contribution

It extends conformal prediction guarantees to non-iid data from dynamical systems using blocking and decoupling techniques, bridging online and offline methods.

Findings

RCPS attains iid-like guarantees under stationary, contractive dynamics.

Guarantees degrade gracefully when data deviates from stationarity.

Tools discussed could unify analysis of online and offline conformal prediction.

Abstract

We study the performance of risk-controlling prediction sets (RCPS), an empirical risk minimization-based formulation of conformal prediction, with a single trajectory of temporally correlated data from an unknown stochastic dynamical system. First, we use the blocking technique to show that RCPS attains performance guarantees similar to those enjoyed in the iid setting whenever data is generated by asymptotically stationary and contractive dynamics. Next, we use the decoupling technique to characterize the graceful degradation in RCPS guarantees when the data generating process deviates from stationarity and contractivity. We conclude by discussing how these tools could be used toward a unified analysis of online and offline conformal prediction algorithms, which are currently treated with very different tools.