Variational Bernstein-von Mises theorem with increasing parameter dimension

TL;DR

This paper develops a finite-sample theoretical framework for Variational Bayes in high-dimensional models, establishing a Bernstein-von Mises theorem and asymptotic properties, with an application to Gaussian mixtures.

Contribution

It extends VB theory to high-dimensional settings with latent variables, providing non-asymptotic guarantees and asymptotic normality results.

Findings

Proves a non-asymptotic variational Bernstein--von Mises theorem.

Establishes consistency and asymptotic normality of VB estimators.

Demonstrates applicability to multivariate Gaussian mixture models.

Abstract

Variational Bayes (VB) provides a computationally efficient alternative to Markov Chain Monte Carlo, especially for high-dimensional and large-scale inference. However, existing theory on VB primarily focuses on fixed-dimensional settings or specific models. To address this limitation, this paper develops a finite-sample theory for VB in a broad class of parametric models with latent variables. We establish theoretical properties of the VB posterior, including a non-asymptotic variational Bernstein--von Mises theorem. Furthermore, we derive consistency and asymptotic normality of the VB estimator. An application to multivariate Gaussian mixture models is presented for illustration.