Variational Neural Networks

TL;DR

This paper introduces Variational Neural Networks, a novel approach for uncertainty estimation that samples layer outputs from Gaussian distributions, outperforming existing single-bin Bayesian methods in uncertainty quality.

Contribution

The paper proposes a new neural network method that samples layer outputs from Gaussian distributions, offering improved uncertainty estimation over traditional Bayesian approaches.

Findings

Achieves better uncertainty quality than Monte Carlo Dropout.

Outperforms Bayes By Backpropagation in uncertainty estimation.

Demonstrates effectiveness in uncertainty quality experiments.

Abstract

Bayesian Neural Networks (BNNs) provide a tool to estimate the uncertainty of a neural network by considering a distribution over weights and sampling different models for each input. In this paper, we propose a method for uncertainty estimation in neural networks which, instead of considering a distribution over weights, samples outputs of each layer from a corresponding Gaussian distribution, parametrized by the predictions of mean and variance sub-layers. In uncertainty quality estimation experiments, we show that the proposed method achieves better uncertainty quality than other single-bin Bayesian Model Averaging methods, such as Monte Carlo Dropout or Bayes By Backpropagation methods.