Evidential Uncertainty Sets in Deep Classifiers Using Conformal Prediction

TL;DR

This paper introduces Evidential Conformal Prediction (ECP), a novel method that leverages evidential deep learning to generate adaptive, reliable conformal prediction sets for image classifiers, improving over existing methods.

Contribution

The paper presents a new ECP method that integrates evidential deep learning with conformal prediction to enhance uncertainty quantification in image classification.

Findings

ECP outperforms three state-of-the-art methods in set size and adaptivity.

ECP maintains coverage of true labels while providing more efficient prediction sets.

Experimental results demonstrate improved uncertainty quantification in deep classifiers.

Abstract

In this paper, we propose Evidential Conformal Prediction (ECP) method for image classifiers to generate the conformal prediction sets. Our method is designed based on a non-conformity score function that has its roots in Evidential Deep Learning (EDL) as a method of quantifying model (epistemic) uncertainty in DNN classifiers. We use evidence that are derived from the logit values of target labels to compute the components of our non-conformity score function: the heuristic notion of uncertainty in CP, uncertainty surprisal, and expected utility. Our extensive experimental evaluation demonstrates that ECP outperforms three state-of-the-art methods for generating CP sets, in terms of their set sizes and adaptivity while maintaining the coverage of true labels.