Loss-Based Variational Bayes Prediction

TL;DR

This paper introduces a new variational Bayes prediction method for high-dimensional models that improves predictive accuracy and robustness to misspecification, with theoretical analysis and empirical validation.

Contribution

It develops a variational approximation to a generalized posterior focused on predictive accuracy, offering a novel approach for high-dimensional Bayesian prediction.

Findings

Provides more accurate predictions than existing methods.

Demonstrates robustness to model misspecification.

Validates approach on simulated and real data.

Abstract

We propose a new approach to Bayesian prediction that caters for models with a large number of parameters and is robust to model misspecification. Given a class of high-dimensional (but parametric) predictive models, this new approach constructs a posterior predictive using a variational approximation to a generalized posterior that is directly focused on predictive accuracy. The theoretical behavior of the new prediction approach is analyzed and a form of optimality demonstrated. Applications to both simulated and empirical data using high-dimensional Bayesian neural network and autoregressive mixture models demonstrate that the approach provides more accurate results than various alternatives, including misspecified likelihood-based predictions.