Linear-cost unbiased posterior estimates for crossed effects and matrix factorization models via couplings

TL;DR

This paper introduces a linear-cost, unbiased MCMC method using couplings for high-dimensional Bayesian models, achieving efficient convergence and accurate posterior estimates in complex matrix factorization and crossed effects models.

Contribution

It develops a novel coupling-based MCMC scheme that provides unbiased estimates with linear computational cost for high-dimensional models, improving convergence speed and accuracy.

Findings

Achieves unbiased posterior estimates with linear cost.

Provides bounds on coalescence times for Gaussian targets.

Demonstrates improved convergence in matrix factorization models.

Abstract

We design and analyze unbiased Markov chain Monte Carlo (MCMC) schemes based on couplings of blocked Gibbs samplers (BGSs), whose total computational costs scale linearly with the number of parameters and data points. Our methodology is designed for and applicable to high-dimensional BGS with conditionally independent blocks, which are often encountered in Bayesian modeling. We provide bounds on the expected number of iterations needed for coalescence for Gaussian targets, as well as on the tails of the coalescence times distribution. These imply that practical two-step coupling strategies achieve coalescence times that match the relaxation times of the original BGS scheme up to logarithmic factors. To illustrate the practical relevance of our methodology, we apply it to high-dimensional crossed random effect and probabilistic matrix factorization models, for which we develop a novel BGS scheme with improved convergence speed. Our methodology provides unbiased posterior estimates at linear cost (usually requiring only a few BGS iterations for problems with thousands of parameters), matching state-of-the-art procedures for both frequentist and Bayesian estimation of those models.