Normalizing Flows for Conformal Regression

TL;DR

This paper introduces a novel method using normalizing flows to improve the efficiency of conformal prediction intervals by localizing calibration based on learned error metrics, applicable to any prediction model.

Contribution

It proposes a general scheme to localize conformal prediction intervals through training a normalizing flow to learn an optimal error metric, enhancing interval efficiency without retraining the prediction model.

Findings

The method effectively localizes prediction intervals based on learned error metrics.

It estimates the gap between nominal and empirical conditional validity.

Compatible with existing locally-adaptive conformal prediction strategies.

Abstract

Conformal Prediction (CP) algorithms estimate the uncertainty of a prediction model by calibrating its outputs on labeled data. The same calibration scheme usually applies to any model and data without modifications. The obtained prediction intervals are valid by construction but could be inefficient, i.e. unnecessarily big, if the prediction errors are not uniformly distributed over the input space. We present a general scheme to localize the intervals by training the calibration process. The standard prediction error is replaced by an optimized distance metric that depends explicitly on the object attributes. Learning the optimal metric is equivalent to training a Normalizing Flow that acts on the joint distribution of the errors and the inputs. Unlike the Error Reweighting CP algorithm of Papadopoulos et al. (2008), the framework allows estimating the gap between nominal and empirical conditional validity. The approach is compatible with existing locally-adaptive CP strategies based on re-weighting the calibration samples and applies to any point-prediction model without retraining.