Variational Inference for Generalized Linear Mixed Models Using Partially Noncentered Parametrizations

TL;DR

This paper explores the use of partially noncentered parametrizations in variational Bayes for generalized linear mixed models, demonstrating improved convergence, accuracy, and model selection capabilities over traditional parametrizations.

Contribution

It introduces an algorithm for nonconjugate variational message passing in GLMMs and shows how partial noncentering adapts to data, enhancing convergence and approximation quality.

Findings

Partial noncentering accelerates convergence.

It yields more accurate posterior estimates.

The variational lower bound aids in model selection.

Abstract

The effects of different parametrizations on the convergence of Bayesian computational algorithms for hierarchical models are well explored. Techniques such as centering, noncentering and partial noncentering can be used to accelerate convergence in MCMC and EM algorithms but are still not well studied for variational Bayes (VB) methods. As a fast deterministic approach to posterior approximation, VB is attracting increasing interest due to its suitability for large high-dimensional data. Use of different parametrizations for VB has not only computational but also statistical implications, as different parametrizations are associated with different factorized posterior approximations. We examine the use of partially noncentered parametrizations in VB for generalized linear mixed models (GLMMs). Our paper makes four contributions. First, we show how to implement an algorithm called nonconjugate variational message passing for GLMMs. Second, we show that the partially noncentered parametrization can adapt to the quantity of information in the data and determine a parametrization close to optimal. Third, we show that partial noncentering can accelerate convergence and produce more accurate posterior approximations than centering or noncentering. Finally, we demonstrate how the variational lower bound, produced as part of the computation, can be useful for model selection.