Stochastic Bayesian Neural Networks

TL;DR

This paper introduces a stochastic Bayesian neural network that maximizes a novel objective called Stochastic Evidence Lower Bound, improving performance and scalability over previous methods on multiple datasets.

Contribution

It proposes a new stochastic variational inference method for Bayesian neural networks using the Stochastic Evidence Lower Bound, enhancing scalability and accuracy.

Findings

Outperforms previous state-of-the-art algorithms on UCI datasets

Achieves better test RMSE and log likelihood scores

Demonstrates scalability to larger datasets

Abstract

Bayesian neural networks perform variational inference over the weights however calculation of the posterior distribution remains a challenge. Our work builds on variational inference techniques for bayesian neural networks using the original Evidence Lower Bound. In this paper, we present a stochastic bayesian neural network in which we maximize Evidence Lower Bound using a new objective function which we name as Stochastic Evidence Lower Bound. We evaluate our network on 5 publicly available UCI datasets using test RMSE and log likelihood as the evaluation metrics. We demonstrate that our work not only beats the previous state of the art algorithms but is also scalable to larger datasets.