Geometric ergodicity of Gibbs samplers for the Horseshoe and its regularized variants

TL;DR

This paper proves geometric ergodicity for Gibbs samplers used with the Horseshoe prior and its variants in Bayesian linear regression, ensuring reliable asymptotic error estimates for these sampling methods.

Contribution

It establishes geometric ergodicity under weaker conditions for the original Horseshoe and its regularized variants, enhancing theoretical understanding of these samplers.

Findings

Proved ergodicity for the original Horseshoe Gibbs sampler under weaker conditions.

Established ergodicity for the regularized Horseshoe prior without truncation constraints.

Confirmed ergodicity for a variant of the regularized Horseshoe prior.

Abstract

The Horseshoe is a widely used and popular continuous shrinkage prior for high-dimensional Bayesian linear regression. Recently, regularized versions of the Horseshoe prior have also been introduced in the literature. Various Gibbs sampling Markov chains have been developed in the literature to generate approximate samples from the corresponding intractable posterior densities. Establishing geometric ergodicity of these Markov chains provides crucial technical justification for the accuracy of asymptotic standard errors for Markov chain based estimates of posterior quantities. In this paper, we establish geometric ergodicity for various Gibbs samplers corresponding to the Horseshoe prior and its regularized variants in the context of linear regression. First, we establish geometric ergodicity of a Gibbs sampler for the original Horseshoe posterior under strictly weaker conditions than existing analyses in the literature. Second, we consider the regularized Horseshoe prior introduced in Piironen and Vehtari (2017), and prove geometric ergodicity for a Gibbs sampling Markov chain to sample from the corresponding posterior without any truncation constraint on the global and local shrinkage parameters. Finally, we consider a variant of this regularized Horseshoe prior introduced in Nishimura and Suchard (2020), and again establish geometric ergodicity for a Gibbs sampling Markov chain to sample from the corresponding posterior.