Multiplicative Normalizing Flows for Variational Bayesian Neural Networks

TL;DR

This paper introduces multiplicative normalizing flows to enhance variational Bayesian neural networks, significantly improving predictive accuracy and uncertainty estimation by augmenting the approximate posterior with flexible transformations.

Contribution

It presents a novel approach that combines multiplicative noise with normalizing flows for better posterior approximation in Bayesian neural networks.

Findings

Improved predictive accuracy over classical mean field methods.

Enhanced uncertainty estimation in Bayesian neural networks.

Efficient implementation allowing local reparametrizations.

Abstract

We reinterpret multiplicative noise in neural networks as auxiliary random variables that augment the approximate posterior in a variational setting for Bayesian neural networks. We show that through this interpretation it is both efficient and straightforward to improve the approximation by employing normalizing flows while still allowing for local reparametrizations and a tractable lower bound. In experiments we show that with this new approximation we can significantly improve upon classical mean field for Bayesian neural networks on both predictive accuracy as well as predictive uncertainty.