Scalable Bayesian neural networks by layer-wise input augmentation

TL;DR

This paper presents a scalable method for Bayesian neural networks that enhances uncertainty estimation by augmenting layer inputs with latent variables, achieving state-of-the-art results in large-scale image classification.

Contribution

It introduces implicit Bayesian neural networks using layer-wise input augmentation, simplifying inference and improving uncertainty quantification in deep learning.

Findings

Achieves state-of-the-art calibration and robustness.

Effective uncertainty characterization on large-scale tasks.

Scalable approach for Bayesian neural networks.

Abstract

We introduce implicit Bayesian neural networks, a simple and scalable approach for uncertainty representation in deep learning. Standard Bayesian approach to deep learning requires the impractical inference of the posterior distribution over millions of parameters. Instead, we propose to induce a distribution that captures the uncertainty over neural networks by augmenting each layer's inputs with latent variables. We present appropriate input distributions and demonstrate state-of-the-art performance in terms of calibration, robustness and uncertainty characterisation over large-scale, multi-million parameter image classification tasks.