Conformal Prediction and Trustworthy AI

TL;DR

This paper reviews how conformal prediction methods can enhance trustworthy AI by providing reliable uncertainty quantification, addressing generalization, bias, and governance issues, supported by experiments and practical examples.

Contribution

It explores the potential of conformal prediction to improve trustworthy AI beyond basic validity, including applications in bias detection and AI governance.

Findings

Conformal predictors offer well-calibrated uncertainty estimates.

They can be used to identify and mitigate bias in AI models.

Experiments demonstrate their effectiveness in real-world scenarios.

Abstract

Conformal predictors are machine learning algorithms developed in the 1990's by Gammerman, Vovk, and their research team, to provide set predictions with guaranteed confidence level. Over recent years, they have grown in popularity and have become a mainstream methodology for uncertainty quantification in the machine learning community. From its beginning, there was an understanding that they enable reliable machine learning with well-calibrated uncertainty quantification. This makes them extremely beneficial for developing trustworthy AI, a topic that has also risen in interest over the past few years, in both the AI community and society more widely. In this article, we review the potential for conformal prediction to contribute to trustworthy AI beyond its marginal validity property, addressing problems such as generalization risk and AI governance. Experiments and examples are also provided to demonstrate its use as a well-calibrated predictor and for bias identification and mitigation.