Quantification of Credal Uncertainty: A Distance-Based Approach

TL;DR

This paper introduces a distance-based framework to quantify different types of uncertainty in credal sets for multiclass classification, providing interpretable, theoretically sound, and computationally efficient measures.

Contribution

It proposes a novel family of uncertainty measures within the IPMs framework, including a multiclass generalization of binary uncertainty measures, with empirical validation.

Findings

Measures are interpretable and satisfy theoretical criteria.

Framework is computationally tractable for common IPMs.

Empirical results show practical effectiveness at low cost.

Abstract

Credal sets, i.e., closed convex sets of probability measures, provide a natural framework to represent aleatoric and epistemic uncertainty in machine learning. Yet how to quantify these two types of uncertainty for a given credal set, particularly in multiclass classification, remains underexplored. In this paper, we propose a distance-based approach to quantify total, aleatoric, and epistemic uncertainty for credal sets. Concretely, we introduce a family of such measures within the framework of Integral Probability Metrics (IPMs). The resulting quantities admit clear semantic interpretations, satisfy natural theoretical desiderata, and remain computationally tractable for common choices of IPMs. We instantiate the framework with the total variation distance and obtain simple, efficient uncertainty measures for multiclass classification. In the binary case, this choice recovers established uncertainty measures, for which a principled multiclass generalization has so far been missing. Empirical results confirm practical usefulness, with favorable performance at low computational cost.