Density estimation in representation space to predict model uncertainty

TL;DR

This paper introduces a simple method to estimate model uncertainty by analyzing data density in the representation space, improving detection of out-of-distribution samples and prediction confidence.

Contribution

It presents a novel approach that uses representation space density to predict model uncertainty, capable of detecting out-of-distribution data without prior examples.

Findings

Effective in in-distribution uncertainty estimation

Accurate out-of-distribution detection

Applicable to state-of-the-art image classifiers

Abstract

Deep learning models frequently make incorrect predictions with high confidence when presented with test examples that are not well represented in their training dataset. We propose a novel and straightforward approach to estimate prediction uncertainty in a pre-trained neural network model. Our method estimates the training data density in representation space for a novel input. A neural network model then uses this information to determine whether we expect the pre-trained model to make a correct prediction. This uncertainty model is trained by predicting in-distribution errors, but can detect out-of-distribution data without having seen any such example. We test our method for a state-of-the art image classification model in the settings of both in-distribution uncertainty estimation as well as out-of-distribution detection.