An out-of-distribution discriminator based on Bayesian neural network epistemic uncertainty

TL;DR

This paper introduces an out-of-distribution detection method using Bayesian neural network epistemic uncertainty, demonstrating its effectiveness in identifying unfamiliar data compared to GAN discriminators.

Contribution

It proposes a novel OoD detection algorithm based on BNN epistemic uncertainty and evaluates its performance against GAN discriminators.

Findings

Epistemic uncertainty is higher for out-of-distribution images.

The proposed OoD detection method performs comparably to GAN discriminators.

Factors influencing OoD detection include training data representation and network architecture.

Abstract

Neural networks have revolutionized the field of machine learning with increased predictive capability. In addition to improving the predictions of neural networks, there is a simultaneous demand for reliable uncertainty quantification on estimates made by machine learning methods such as neural networks. Bayesian neural networks (BNNs) are an important type of neural network with built-in capability for quantifying uncertainty. This paper discusses aleatoric and epistemic uncertainty in BNNs and how they can be calculated. With an example dataset of images where the goal is to identify the amplitude of an event in the image, it is shown that epistemic uncertainty tends to be lower in images which are well-represented in the training dataset and tends to be high in images which are not well-represented. An algorithm for out-of-distribution (OoD) detection with BNN epistemic uncertainty is introduced along with various experiments demonstrating factors influencing the OoD detection capability in a BNN. The OoD detection capability with epistemic uncertainty is shown to be comparable to the OoD detection in the discriminator network of a generative adversarial network (GAN) with comparable network architecture.