# Correlated Parameters to Accurately Measure Uncertainty in Deep Neural Networks

## TL;DR

This paper introduces a Bayesian deep learning method using correlated variational parameters to improve uncertainty estimation and robustness against overfitting, demonstrated on MNIST and CIFAR-10 datasets.

## Contribution

It proposes a novel variational inference approach with correlated parameters for Bayesian neural networks, reducing complexity while enhancing uncertainty quantification.

## Key findings

- Effective uncertainty estimation demonstrated on benchmark datasets
- Robustness to overfitting shown in experiments
- Fewer parameters needed compared to traditional Bayesian methods

## Abstract

In this article a novel approach for training deep neural networks using Bayesian techniques is presented. The Bayesian methodology allows for an easy evaluation of model uncertainty and additionally is robust to overfitting. These are commonly the two main problems classical, i.e. non-Bayesian, architectures have to struggle with. The proposed approach applies variational inference in order to approximate the intractable posterior distribution. In particular, the variational distribution is defined as product of multiple multivariate normal distributions with tridiagonal covariance matrices. Each single normal distribution belongs either to the weights, or to the biases corresponding to one network layer. The layer-wise a posteriori variances are defined based on the corresponding expectation values and further the correlations are assumed to be identical. Therefore, only a few additional parameters need to be optimized compared to non-Bayesian settings. The novel approach is successfully evaluated on basis of the popular benchmark datasets MNIST and CIFAR-10.