Bayesian Autoencoders: Analysing and Fixing the Bernoulli likelihood for Out-of-Distribution Detection

TL;DR

This paper investigates why Bernoulli likelihoods in autoencoders can fail for out-of-distribution detection and proposes Bayesian methods and alternative distributions to improve OOD detection accuracy.

Contribution

It identifies the limitations of Bernoulli likelihoods in autoencoders for OOD detection and introduces Bayesian autoencoders and alternative likelihood models as solutions.

Findings

Bernoulli likelihoods can produce high likelihoods for OOD inputs.

Bayesian autoencoders provide uncertainty estimates that improve OOD detection.

Alternative likelihood distributions enhance the reliability of OOD detection.

Abstract

After an autoencoder (AE) has learnt to reconstruct one dataset, it might be expected that the likelihood on an out-of-distribution (OOD) input would be low. This has been studied as an approach to detect OOD inputs. Recent work showed this intuitive approach can fail for the dataset pairs FashionMNIST vs MNIST. This paper suggests this is due to the use of Bernoulli likelihood and analyses why this is the case, proposing two fixes: 1) Compute the uncertainty of likelihood estimate by using a Bayesian version of the AE. 2) Use alternative distributions to model the likelihood.