Frequentist Oracle Properties of Bayesian Stacking Estimators

TL;DR

This paper establishes frequentist oracle properties for Bayesian stacking estimators, demonstrating their asymptotic optimality and potential to outperform individual models through theoretical proofs and simulations.

Contribution

It provides the first frequentist oracle property proofs for Bayesian stacking estimators, bridging Bayesian methods with frequentist asymptotic analysis.

Findings

Bayesian stacking estimators satisfy oracle properties asymptotically.

Stacking can outperform the best individual candidate model.

Theoretical results are supported by Monte Carlo experiments.

Abstract

Compromise estimation entails using a weighted average of outputs from several candidate models, and is a viable alternative to model selection when the choice of model is not obvious. As such, it is a tool used by both frequentists and Bayesians, and in both cases, the literature is vast and includes studies of performance in simulations and applied examples. However, frequentist researchers often prove oracle properties, showing that a proposed average asymptotically performs at least as well as any other average comprising the same candidates. On the Bayesian side, such oracle properties are yet to be established. This paper considers Bayesian stacking estimators, and evaluates their performance using frequentist asymptotics. Oracle properties are derived for estimators stacking Bayesian linear and logistic regression models, and combined with Monte Carlo experiments that show Bayesian stacking may outperform the best candidate model included in the stack. Thus, the result is not only a frequentist motivation of a fundamentally Bayesian procedure, but also an extended range of methods available to frequentist practitioners.