Uncertainty Estimations by Softplus normalization in Bayesian Convolutional Neural Networks with Variational Inference

TL;DR

This paper proposes a new method for estimating uncertainty in Bayesian CNNs by normalizing Softplus outputs, enabling coherent aleatoric and epistemic uncertainty estimation with performance comparable to traditional methods.

Contribution

It introduces a Softplus normalization technique for uncertainty estimation in Bayesian CNNs using variational inference, applicable across various architectures and datasets.

Findings

Achieves uncertainty estimation comparable to frequentist inference.

Provides a coherent measure for aleatoric and epistemic uncertainties.

Demonstrates applicability across multiple datasets (MNIST, CIFAR-10, CIFAR-100).

Abstract

We introduce a novel uncertainty estimation for classification tasks for Bayesian convolutional neural networks with variational inference. By normalizing the output of a Softplus function in the final layer, we estimate aleatoric and epistemic uncertainty in a coherent manner. The intractable posterior probability distributions over weights are inferred by Bayes by Backprop. Firstly, we demonstrate how this reliable variational inference method can serve as a fundamental construct for various network architectures. On multiple datasets in supervised learning settings (MNIST, CIFAR-10, CIFAR-100), this variational inference method achieves performances equivalent to frequentist inference in identical architectures, while the two desiderata, a measure for uncertainty and regularization are incorporated naturally. Secondly, we examine how our proposed measure for aleatoric and epistemic uncertainties is derived and validate it on the aforementioned datasets.