Conformal calibrators

TL;DR

This paper introduces fully adaptive split- and cross-conformal predictive systems that calibrate existing models to ensure probability calibration, even without IID assumptions.

Contribution

It develops calibration methods for conformal predictive systems that are fully adaptive and do not require the predictive system to be valid beforehand.

Findings

Guarantees calibrated probability outputs

Works without IID assumptions

Applicable to existing predictive systems

Abstract

Most existing examples of full conformal predictive systems, split-conformal predictive systems, and cross-conformal predictive systems impose severe restrictions on the adaptation of predictive distributions to the test object at hand. In this paper we develop split-conformal and cross-conformal predictive systems that are fully adaptive. Our method consists in calibrating existing predictive systems; the input predictive system is not supposed to satisfy any properties of validity, whereas the output predictive system is guaranteed to be calibrated in probability. It is interesting that the method may also work without the IID assumption, standard in conformal prediction.