Affine Invariant Ensemble Transform Methods to Improve Predictive Uncertainty in Neural Networks

TL;DR

This paper introduces affine invariant ensemble transform methods to enhance predictive uncertainty estimation in neural networks through Bayesian inference, utilizing particle systems with proven convergence properties.

Contribution

It proposes novel affine invariant ensemble transform techniques for Bayesian neural network approximation, with theoretical convergence guarantees.

Findings

Effective Bayesian approximation for neural networks.

Quantitative convergence rates of particle systems.

Improved uncertainty quantification in predictions.

Abstract

We consider the problem of performing Bayesian inference for logistic regression using appropriate extensions of the ensemble Kalman filter. Two interacting particle systems are proposed that sample from an approximate posterior and prove quantitative convergence rates of these interacting particle systems to their mean-field limit as the number of particles tends to infinity. Furthermore, we apply these techniques and examine their effectiveness as methods of Bayesian approximation for quantifying predictive uncertainty in neural networks.