Conditional validity of heteroskedastic conformal regression

TL;DR

This paper investigates how conformal prediction methods can be adapted to heteroskedastic data, providing theoretical insights and experimental comparisons to ensure valid, adaptive prediction intervals.

Contribution

It introduces and analyzes methods like normalized and Mondrian conformal prediction for heteroskedastic data, linking their validity to distributional assumptions.

Findings

Normalized and Mondrian conformal methods improve interval adaptivity.

Conditional validity depends on assumptions about data distribution.

Experimental results compare different conformal approaches systematically.

Abstract

Conformal prediction, and split conformal prediction as a specific implementation, offer a distribution-free approach to estimating prediction intervals with statistical guarantees. Recent work has shown that split conformal prediction can produce state-of-the-art prediction intervals when focusing on marginal coverage, i.e. on a calibration dataset the method produces on average prediction intervals that contain the ground truth with a predefined coverage level. However, such intervals are often not adaptive, which can be problematic for regression problems with heteroskedastic noise. This paper tries to shed new light on how prediction intervals can be constructed, using methods such as normalized and Mondrian conformal prediction, in such a way that they adapt to the heteroskedasticity of the underlying process. Theoretical and experimental results are presented in which these methods are compared in a systematic way. In particular, it is shown how the conditional validity of a chosen conformal predictor can be related to (implicit) assumptions about the data-generating distribution.