Online conformal prediction with decaying step sizes

TL;DR

This paper presents an online conformal prediction method with decaying step sizes that guarantees coverage, estimates population quantiles, and improves practical stability of coverage in stable distributions.

Contribution

It introduces a novel online conformal prediction approach that simultaneously guarantees coverage and estimates population quantiles with improved stability.

Findings

Coverage remains close to the desired level in stable distributions.

The method provides simultaneous coverage guarantees and quantile estimation.

Experimental results show improved practical properties over previous methods.

Abstract

We introduce a method for online conformal prediction with decaying step sizes. Like previous methods, ours possesses a retrospective guarantee of coverage for arbitrary sequences. However, unlike previous methods, we can simultaneously estimate a population quantile when it exists. Our theory and experiments indicate substantially improved practical properties: in particular, when the distribution is stable, the coverage is close to the desired level for every time point, not just on average over the observed sequence.