Coupling-based convergence assessment of some Gibbs samplers for high-dimensional Bayesian regression with shrinkage priors

TL;DR

This paper introduces coupling techniques for Gibbs samplers in high-dimensional Bayesian regression, providing practical convergence diagnostics and demonstrating that fewer than 1000 iterations can suffice for large-scale problems.

Contribution

The authors develop novel coupling methods with geometric drift conditions for high-dimensional Gibbs samplers, enabling efficient convergence assessment without long-run diagnostics.

Findings

Less than 1000 iterations needed for convergence in 100,000 covariate regression

Couplings are scalable and effective for high-dimensional settings

Prior choice impacts computational efficiency and convergence speed

Abstract

We consider Markov chain Monte Carlo (MCMC) algorithms for Bayesian high-dimensional regression with continuous shrinkage priors. A common challenge with these algorithms is the choice of the number of iterations to perform. This is critical when each iteration is expensive, as is the case when dealing with modern data sets, such as genome-wide association studies with thousands of rows and up to hundred of thousands of columns. We develop coupling techniques tailored to the setting of high-dimensional regression with shrinkage priors, which enable practical, non-asymptotic diagnostics of convergence without relying on traceplots or long-run asymptotics. By establishing geometric drift and minorization conditions for the algorithm under consideration, we prove that the proposed couplings have finite expected meeting time. Focusing on a class of shrinkage priors which includes the 'Horseshoe', we empirically demonstrate the scalability of the proposed couplings. A highlight of our findings is that less than 1000 iterations can be enough for a Gibbs sampler to reach stationarity in a regression on 100,000 covariates. The numerical results also illustrate the impact of the prior on the computational efficiency of the coupling, and suggest the use of priors where the local precisions are Half-t distributed with degree of freedom larger than one.