Safe, Scalable, and Accurate Bayes Posterior Sampling for Large-Data Generalized Linear Mixed Models

TL;DR

This paper introduces a novel stochastic mirror Langevin dynamics algorithm for scalable Bayesian inference in large-data generalized linear mixed models, addressing issues with existing methods.

Contribution

It proposes a new stochastic mirror Langevin dynamics algorithm with guidelines and a post-processing step for accurate posterior variance estimation in large datasets.

Findings

The new method outperforms existing stochastic gradient methods in accuracy.

The post-processing step corrects variance estimation bias due to subsampling.

Empirical results demonstrate improved scalability and reliability.

Abstract

We consider the problem of scalable sampling algorithms to fit Bayesian generalized linear mixed models on large datasets. Stochastic gradient Langevin dynamics, coupled with smooth re-parameterizations of variance parameters, produces divergent Markov chains and cannot be reliably used for sampling covariance parameters of random effects. We advocate the use of a mirror Langevin dynamics algorithm, propose the novel stochastic mirror Langevin dynamics based on data subsampling, and provide concrete guidelines for its use in a Bayesian inference framework. Based on an explicit Wasserstein distance error bound between the posterior and its algorithmic approximation, we propose a post-processing step that yields an asymptotic, order-wise correct estimation of the posterior variance, eliminating the irreducible posterior variance estimation bias due to subsampling. Empirical performance of the method is evaluated through simulated experiments and a longitudinal study of pain trajectories in a study of breast cancer survivors.