Regenerative Simulation for the Bayesian Lasso

TL;DR

This paper introduces a regenerative simulation approach for the Bayesian Lasso Gibbs sampler, providing a theoretically sound method for variance estimation and assessing convergence, which improves upon existing ad-hoc diagnostics.

Contribution

It identifies regenerative structure in the Bayesian Lasso Gibbs sampler, enabling rigorous variance estimation and convergence assessment.

Findings

The regenerative method yields consistent variance estimates.

It provides an upper bound on the total variation distance.

Numerical validation shows the method's effectiveness.

Abstract

The Gibbs sampler of Park and Casella is one of the most popular MCMC methods for sampling from the posterior density of the Bayesian Lasso regression. As with many Markov chain samplers, their Gibbs sampler lacks a theoretically sound method of output analysis --- a method for estimating the variance of a given ergodic average and estimating how closely the chain is sampling from the stationary distribution, that is, the burn-in. In this paper, we address this shortcoming by identifying regenerative structure in the sampler of Park and Casella, thus providing a theoretically sound method of assessing its performance. The regenerative structure provides both a strongly consistent variance estimator, and an estimator of (an upper bound on) the total variation distance from the target posterior density. The result is a simple and theoretically sound way to assess the stationarity of the Park and Casella and, more generally, other MCMC samplers, for which regenerative simulation is possible. We perform a numerical study in which we validate the standard errors calculated by our regenerative method by comparing it with the standard errors calculated by an AR(1) heuristic approximation. Thus, we show that for the Bayesian Lasso model, the regenerative method is a viable and theoretically justified alternative to the existing ad-hoc MCMC convergence diagnostics.