What is Flagged in Uncertainty Quantification? Latent Density Models for Uncertainty Categorization

TL;DR

This paper introduces a framework for categorizing uncertain examples flagged by UQ methods in classification, using a confusion density matrix to distinguish OOD, boundary, and high-misclassification regions, enhancing assessment of UQ methods.

Contribution

The paper proposes a novel confusion density matrix framework for categorizing uncertain examples in classification, providing a new perspective for evaluating UQ methods.

Findings

Effective categorization of suspicious examples into OOD, boundary, and high-misclassification classes.

Framework offers a new perspective for comparing different UQ methods.

Provides a valuable benchmark for UQ assessment.

Abstract

Uncertainty Quantification (UQ) is essential for creating trustworthy machine learning models. Recent years have seen a steep rise in UQ methods that can flag suspicious examples, however, it is often unclear what exactly these methods identify. In this work, we propose a framework for categorizing uncertain examples flagged by UQ methods in classification tasks. We introduce the confusion density matrix -- a kernel-based approximation of the misclassification density -- and use this to categorize suspicious examples identified by a given uncertainty method into three classes: out-of-distribution (OOD) examples, boundary (Bnd) examples, and examples in regions of high in-distribution misclassification (IDM). Through extensive experiments, we show that our framework provides a new and distinct perspective for assessing differences between uncertainty quantification methods, thereby forming a valuable assessment benchmark.