Logit Disagreement: OoD Detection with Bayesian Neural Networks

TL;DR

This paper introduces a novel method for out-of-distribution detection using Bayesian neural networks by measuring logit disagreement, which improves epistemic uncertainty estimation and outperforms existing mutual information metrics.

Contribution

It proposes a new approach to estimate epistemic uncertainty through logit disagreement, demonstrating improved OoD detection performance over mutual information.

Findings

Logit disagreement scores outperform mutual information in OoD detection.

Epistemic uncertainty scores perform comparably to predictive entropy on benchmark datasets.

The method enhances uncertainty quantification in Bayesian neural networks.

Abstract

Bayesian neural networks (BNNs), which estimate the full posterior distribution over model parameters, are well-known for their role in uncertainty quantification and its promising application in out-of-distribution detection (OoD). Amongst other uncertainty measures, BNNs provide a state-of-the art estimation of predictive entropy (total uncertainty) which can be decomposed as the sum of mutual information and expected entropy. In the context of OoD detection the estimation of predictive uncertainty in the form of the predictive entropy score confounds aleatoric and epistemic uncertainty, the latter being hypothesized to be high for OoD points. Despite these justifications, the mutual information score has been shown to perform worse than predictive entropy. Taking inspiration from Bayesian variational autoencoder (BVAE) literature, this work proposes to measure the disagreement between a corrected version of the pre-softmax quantities, otherwise known as logits, as an estimate of epistemic uncertainty for Bayesian NNs under mean field variational inference. The three proposed epistemic uncertainty scores demonstrate marked improvements over mutual information on a range of OoD experiments, with equal performance otherwise. Moreover, the epistemic uncertainty scores perform on par with the Bayesian benchmark predictive entropy on a range of MNIST and CIFAR10 experiments.