Black Box Variational Bayesian Model Averaging

TL;DR

This paper introduces a 'black box' variational Bayesian approach to model averaging that simplifies implementation and broadens applicability, providing an efficient alternative to traditional methods for handling model uncertainty.

Contribution

It presents a novel variational Bayesian inference method for Bayesian Model Averaging that is easy to apply across different models without extensive derivation.

Findings

Effective in multiple example scenarios

Avoids complex posterior sampling procedures

Easily adaptable to various models

Abstract

For many decades now, Bayesian Model Averaging (BMA) has been a popular framework to systematically account for model uncertainty that arises in situations when multiple competing models are available to describe the same or similar physical process. The implementation of this framework, however, comes with a multitude of practical challenges including posterior approximation via Markov Chain Monte Carlo and numerical integration. We present a Variational Bayesian Inference approach to BMA as a viable alternative to the standard solutions which avoids many of the aforementioned pitfalls. The proposed method is "black box" in the sense that it can be readily applied to many models with little to no model-specific derivation. We illustrate the utility of our variational approach on a suite of examples and discuss all the necessary implementation details. Fully documented Python code with all the examples is provided as well.