Conformal Prediction: A Data Perspective

TL;DR

This survey reviews conformal prediction, a distribution-free uncertainty quantification method, highlighting recent advances and challenges in applying it to diverse, complex, and large-scale data and models.

Contribution

It provides a comprehensive overview of conformal prediction's foundational concepts and recent data-centric developments across various data modalities.

Findings

Conformal prediction offers valid predictive inference across diverse data types.

Recent methods extend CP to structured, unstructured, and dynamic data.

Challenges include scalability and applicability to large-scale models.

Abstract

Conformal prediction (CP), a distribution-free uncertainty quantification (UQ) framework, reliably provides valid predictive inference for black-box models. CP constructs prediction sets that contain the true output with a specified probability. However, modern data science diverse modalities, along with increasing data and model complexity, challenge traditional CP methods. These developments have spurred novel approaches to address evolving scenarios. This survey reviews the foundational concepts of CP and recent advancements from a data-centric perspective, including applications to structured, unstructured, and dynamic data. We also discuss the challenges and opportunities CP faces in large-scale data and models.