A novel Deep Learning approach for one-step Conformal Prediction approximation

TL;DR

This paper introduces a new deep learning-based conformal loss function that approximates the two-step conformal prediction process in a single step, reducing training time while maintaining accuracy.

Contribution

It proposes a novel conformal loss function enabling one-step approximation of conformal prediction within deep learning models, improving efficiency.

Findings

Achieves up to 86% training time reduction.

Maintains comparable validity and predictive efficiency.

Performs well across multiple datasets and prediction tasks.

Abstract

Deep Learning predictions with measurable confidence are increasingly desirable for real-world problems, especially in high-risk settings. The Conformal Prediction (CP) framework is a versatile solution that guarantees a maximum error rate given minimal constraints. In this paper, we propose a novel conformal loss function that approximates the traditionally two-step CP approach in a single step. By evaluating and penalising deviations from the stringent expected CP output distribution, a Deep Learning model may learn the direct relationship between the input data and the conformal p-values. We carry out a comprehensive empirical evaluation to show our novel loss function's competitiveness for seven binary and multi-class prediction tasks on five benchmark datasets. On the same datasets, our approach achieves significant training time reductions up to 86% compared to Aggregated Conformal Prediction (ACP), while maintaining comparable approximate validity and predictive efficiency.