Bayesian Neural Networks

TL;DR

Bayesian neural networks incorporate uncertainty quantification into neural network predictions, addressing biases and limitations of traditional methods by characterizing inherent and epistemic uncertainties.

Contribution

This paper introduces a Bayesian framework for neural networks, detailing methods to quantify and analyze uncertainty, and discusses practical pitfalls of existing techniques.

Findings

Methods to quantify prediction errors in Bayesian neural networks

Identification of pitfalls in current uncertainty estimation techniques

Highlighting the need for improved statistical inference methods

Abstract

In recent times, neural networks have become a powerful tool for the analysis of complex and abstract data models. However, their introduction intrinsically increases our uncertainty about which features of the analysis are model-related and which are due to the neural network. This means that predictions by neural networks have biases which cannot be trivially distinguished from being due to the true nature of the creation and observation of data or not. In order to attempt to address such issues we discuss Bayesian neural networks: neural networks where the uncertainty due to the network can be characterised. In particular, we present the Bayesian statistical framework which allows us to categorise uncertainty in terms of the ingrained randomness of observing certain data and the uncertainty from our lack of knowledge about how data can be created and observed. In presenting such techniques we show how errors in prediction by neural networks can be obtained in principle, and provide the two favoured methods for characterising these errors. We will also describe how both of these methods have substantial pitfalls when put into practice, highlighting the need for other statistical techniques to truly be able to do inference when using neural networks.