Distribution-informed Online Conformal Prediction

TL;DR

This paper introduces Conformal Optimistic Prediction (COP), an online conformal prediction method that adapts to data patterns to produce tighter, valid prediction sets even under distribution shifts and adversarial conditions.

Contribution

The paper proposes COP, a novel online conformal prediction algorithm that incorporates data patterns into updates, achieving tighter prediction sets with valid coverage guarantees.

Findings

COP produces shorter prediction intervals than baselines.

COP maintains valid coverage under distribution shifts.

Experimental results confirm the effectiveness of COP.

Abstract

Conformal prediction provides a pivotal and flexible technique for uncertainty quantification by constructing prediction sets with a predefined coverage rate. Many online conformal prediction methods have been developed to address data distribution shifts in fully adversarial environments, resulting in overly conservative prediction sets. We propose Conformal Optimistic Prediction (COP), an online conformal prediction algorithm incorporating underlying data pattern into the update rule. Through estimated cumulative distribution function of non-conformity scores, COP produces tighter prediction sets when predictable pattern exists, while retaining valid coverage guarantees even when estimates are inaccurate. We establish a joint bound on coverage and regret, which further confirms the validity of our approach. We also prove that COP achieves distribution-free, finite-sample coverage under arbitrary learning rates and can converge when scores are $i.i.d.$. The experimental results also show that COP can achieve valid coverage and construct shorter prediction intervals than other baselines.