Variational Bayes in State Space Models: Inferential and Predictive Accuracy

TL;DR

This paper evaluates the accuracy of variational Bayes methods in state space models, revealing that methods not approximating states are more accurate for fixed parameters, but predictive accuracy discrepancies vary over time.

Contribution

It provides a comprehensive theoretical and numerical comparison of variational Bayes methods, highlighting when predictive accuracy may be affected by inferential differences.

Findings

Methods not approximating states outperform others in fixed parameter accuracy.

Predictive accuracy discrepancies are small over short periods but can grow over longer horizons.

The invariance of predictive results to inferential inaccuracies is not always valid.

Abstract

Using theoretical and numerical results, we document the accuracy of commonly applied variational Bayes methods across a range of state space models. The results demonstrate that, in terms of accuracy on fixed parameters, there is a clear hierarchy in terms of the methods, with approaches that do not approximate the states yielding superior accuracy over methods that do. We also document numerically that the inferential discrepancies between the various methods often yield only small discrepancies in predictive accuracy over small out-of-sample evaluation periods. Nevertheless, in certain settings, these predictive discrepancies can become meaningful over a longer out-of-sample period. This finding indicates that the invariance of predictive results to inferential inaccuracy, which has been an oft-touted point made by practitioners seeking to justify the use of variational inference, is not ubiquitous and must be assessed on a case-by-case basis.