Conformal Predictions under Markovian Data

TL;DR

This paper analyzes the split Conformal Prediction method for Markovian data, quantifying how data correlations affect coverage and proposing a thinning technique to mitigate this impact, with theoretical bounds and empirical validation.

Contribution

It provides theoretical bounds on coverage gaps for conformal prediction under Markovian data and introduces K-split CP, an adaptive method to reduce the coverage gap based on mixing properties.

Findings

Coverage gap scales as √(t_mix ln(n)/n)

Thinning calibration data reduces coverage gap to t_mix/(n ln(n))

Proposed methods perform well on synthetic and real datasets

Abstract

We study the split Conformal Prediction method when applied to Markovian data. We quantify the gap in terms of coverage induced by the correlations in the data (compared to exchangeable data). This gap strongly depends on the mixing properties of the underlying Markov chain, and we prove that it typically scales as $\sqrt{t_\mathrm{mix}\ln(n)/n}$ (where $t_\mathrm{mix}$ is the mixing time of the chain). We also derive upper bounds on the impact of the correlations on the size of the prediction set. Finally we present $K$-split CP, a method that consists in thinning the calibration dataset and that adapts to the mixing properties of the chain. Its coverage gap is reduced to $t_\mathrm{mix}/(n\ln(n))$ without really affecting the size of the prediction set. We finally test our algorithms on synthetic and real-world datasets.