Frequentist Asymptotics of Variational Laplace

TL;DR

This paper investigates the theoretical properties of variational Laplace, showing it produces consistent and efficient point estimates under certain conditions, and analyzes its convergence behavior in a frequentist asymptotic framework.

Contribution

It provides the first systematic frequentist asymptotic analysis of variational Laplace, establishing conditions for consistency, efficiency, and distribution convergence.

Findings

Point estimates are asymptotically consistent and efficient.

Conditions for convergence and desirable properties are derived.

Simulation experiments support theoretical results.

Abstract

Variational inference is a general framework to obtain approximations to the posterior distribution in a Bayesian context. In essence, variational inference entails an optimization over a given family of probability distributions to choose the member of this family best approximating the posterior. Variational Laplace, an iterative update scheme motivated by this objective, is widely used in different contexts in the cognitive neuroscience community. However, until now, the theoretical properties of this scheme have not been systematically investigated. Here, we study variational Laplace in the light of frequentist asymptotic statistics. Asymptotical frequentist theory enables one to judge the quality of point estimates by their limit behaviour. We apply this framework to find that point estimates generated by variational Laplace enjoy the desirable properties of asymptotic consistency and efficiency in two toy examples. Furthermore, we derive conditions that are sufficient to establish these properties in a general setting. Besides of point estimates, we also study the frequentist convergence of distributions in the sense of total variation distance, which may be useful to relate variational Laplace both to recent findings regarding variational inference as well as to classical frequentist considerations on the Bayesian posterior. Finally, to illustrate the validity of our theoretical considerations, we conduct simulation experiments in our study examples.