From Risk to Uncertainty: Generating Predictive Uncertainty Measures via Bayesian Estimation

TL;DR

This paper presents a Bayesian framework that decomposes predictive risk into aleatoric and epistemic uncertainties, enabling the generation of various uncertainty measures for improved out-of-distribution and misclassification detection.

Contribution

It introduces a novel decomposition-based framework that unifies different predictive uncertainty measures using Bayesian approximation methods.

Findings

Uncertainty measures effectively detect out-of-distribution instances.

The framework improves misclassification detection performance.

Experimental validation on image datasets confirms usefulness.

Abstract

There are various measures of predictive uncertainty in the literature, but their relationships to each other remain unclear. This paper uses a decomposition of statistical pointwise risk into components, associated with different sources of predictive uncertainty, namely aleatoric uncertainty (inherent data variability) and epistemic uncertainty (model-related uncertainty). Together with Bayesian methods, applied as an approximation, we build a framework that allows one to generate different predictive uncertainty measures. We validate our method on image datasets by evaluating its performance in detecting out-of-distribution and misclassified instances using the AUROC metric. The experimental results confirm that the measures derived from our framework are useful for the considered downstream tasks.