Fast Monte Carlo Markov chains for Bayesian shrinkage models with random effects

TL;DR

This paper introduces a new, efficient MCMC algorithm for Bayesian linear mixed models with shrinkage priors, demonstrating geometric ergodicity even in high-dimensional settings where p >> n.

Contribution

It develops a simple, two-component MCMC algorithm for complex Bayesian models with normal-gamma priors, proving its geometric ergodicity in practical high-dimensional cases.

Findings

The algorithm is geometrically ergodic in most practical settings.

It performs well in high-dimensional scenarios where p exceeds n.

The method improves sampling efficiency for Bayesian shrinkage models.

Abstract

When performing Bayesian data analysis using a general linear mixed model, the resulting posterior density is almost always analytically intractable. However, if proper conditionally conjugate priors are used, there is a simple two-block Gibbs sampler that is geometrically ergodic in nearly all practical settings, including situations where $p > n$ (Abrahamsen and Hobert, 2017). Unfortunately, the (conditionally conjugate) multivariate normal prior on $\beta$ does not perform well in the high-dimensional setting where $p \gg n$. In this paper, we consider an alternative model in which the multivariate normal prior is replaced by the normal-gamma shrinkage prior developed by Griffin and Brown (2010). This change leads to a much more complex posterior density, and we develop a simple MCMC algorithm for exploring it. This algorithm, which has both deterministic and random scan components, is easier to analyze than the more obvious three-step Gibbs sampler. Indeed, we prove that the new algorithm is geometrically ergodic in most practical settings.