Randomness, exchangeability, and conformal prediction

TL;DR

This paper explores the relationship between randomness, exchangeability, and conformal prediction, advocating for a broader use of the functional theory of randomness to understand data assumptions in machine learning.

Contribution

It translates existing results on conformal prediction into the functional theory of randomness, showing how IID-based predictors can be adapted to conformal predictors efficiently.

Findings

Conformal prediction is a universal method under IID assumptions.

Every valid IID confidence predictor can be transformed into a conformal predictor.

The translation enhances practical understanding of conformal prediction's theoretical foundations.

Abstract

This paper argues for a wider use of the functional theory of randomness, a modification of the algorithmic theory of randomness getting rid of unspecified additive constants. Both theories are useful for understanding relationships between the assumptions of IID data and data exchangeability. While the assumption of IID data is standard in machine learning, conformal prediction relies on data exchangeability. Nouretdinov, V'yugin, and Gammerman showed, using the language of the algorithmic theory of randomness, that conformal prediction is a universal method under the assumption of IID data. In this paper (written for the Alex Gammerman Festschrift) I will selectively review connections between exchangeability and the property of being IID, early history of conformal prediction, my encounters and collaboration with Alex and other interesting people, and a translation of Nouretdinov et al.'s results into the language of the functional theory of randomness, which moves it closer to practice. Namely, the translation says that every confidence predictor that is valid for IID data can be transformed to a conformal predictor without losing much in predictive efficiency.