Geometric Ergodicity of Gibbs Samplers in Bayesian Penalized Regression Models

TL;DR

This paper proves that Gibbs samplers for three Bayesian penalized regression models are geometrically ergodic, ensuring reliable Monte Carlo estimates and enabling default starting values, regardless of problem dimension.

Contribution

It establishes geometric ergodicity for Gibbs samplers in Bayesian fused, group, and sparse group lasso models, a novel theoretical result in high-dimensional Bayesian regression.

Findings

Gibbs samplers are geometrically ergodic for these models.

Ensures validity of Monte Carlo estimates via CLT.

Provides default starting values for the samplers.

Abstract

We consider three Bayesian penalized regression models and show that the respective deterministic scan Gibbs samplers are geometrically ergodic regardless of the dimension of the regression problem. We prove geometric ergodicity of the Gibbs samplers for the Bayesian fused lasso, the Bayesian group lasso, and the Bayesian sparse group lasso. Geometric ergodicity along with a moment condition results in the existence of a Markov chain central limit theorem for Monte Carlo averages and ensures reliable output analysis. Our results of geometric ergodicity allow us to also provide default starting values for the Gibbs samplers.