Logit-based Uncertainty Measure in Classification

TL;DR

This paper proposes a new logit-based uncertainty measure for classification that outperforms existing methods in tasks like out-of-sample detection and error identification, with theoretical analysis and practical applications.

Contribution

It introduces a novel, reliable uncertainty measure based on neural network logits, with theoretical insights and demonstrated advantages over existing measures.

Findings

Superior performance in out-of-sample detection

Effective in identifying erroneous predictions

Applicable to GAN training outputs

Abstract

We introduce a new, reliable, and agnostic uncertainty measure for classification tasks called logit uncertainty. It is based on logit outputs of neural networks. We in particular show that this new uncertainty measure yields a superior performance compared to existing uncertainty measures on different tasks, including out of sample detection and finding erroneous predictions. We analyze theoretical foundations of the measure and explore a relationship with high density regions. We also demonstrate how to test uncertainty using intermediate outputs in training of generative adversarial networks. We propose two potential ways to utilize logit-based uncertainty in real world applications, and show that the uncertainty measure outperforms.