Adaptive Conformal Inference by Betting

TL;DR

This paper introduces a parameter-free adaptive conformal inference method that maintains coverage guarantees without relying on data exchangeability or extensive parameter tuning.

Contribution

It proposes a novel approach using online convex optimization techniques for adaptive conformal inference, eliminating the need for learning rate tuning.

Findings

Controls long-term miscoverage at a nominal level

Demonstrates strong empirical performance

Avoids cumbersome parameter tuning

Abstract

Conformal prediction is a valuable tool for quantifying predictive uncertainty of machine learning models. However, its applicability relies on the assumption of data exchangeability, a condition which is often not met in real-world scenarios. In this paper, we consider the problem of adaptive conformal inference without any assumptions about the data generating process. Existing approaches for adaptive conformal inference are based on optimizing the pinball loss using variants of online gradient descent. A notable shortcoming of such approaches is in their explicit dependence on and sensitivity to the choice of the learning rates. In this paper, we propose a different approach for adaptive conformal inference that leverages parameter-free online convex optimization techniques. We prove that our method controls long-term miscoverage frequency at a nominal level and demonstrate its convincing empirical performance without any need of performing cumbersome parameter tuning.