Bayesian Aggregation

TL;DR

This paper discusses Bayesian model aggregation methods, specifically BMA and stacking, comparing their performance, theoretical foundations, and practical applications for improved predictive stability in statistical modeling.

Contribution

It provides a comprehensive comparison of Bayesian model averaging and stacking, including their theoretical properties, implementation details, and extensions for complex models.

Findings

Bayesian stacking often outperforms BMA in predictive accuracy.

Both methods have solid theoretical justifications and probabilistic interpretations.

Extensions enable application to complex and high-dimensional models.

Abstract

A general challenge in statistics is prediction in the presence of multiple candidate models or learning algorithms. Model aggregation tries to combine all predictive distributions from individual models, which is more stable and flexible than single model selection. In this article we describe when and how to aggregate models under the lens of Bayesian decision theory. Among two widely used methods, Bayesian model averaging (BMA) and Bayesian stacking, we compare their predictive performance, and review their theoretical optimality, probabilistic interpretation, practical implementation, and extensions in complex models.