Convergence analysis of block Gibbs samplers for Bayesian linear mixed models with $p>N$

TL;DR

This paper extends the convergence analysis of block Gibbs samplers for Bayesian linear mixed models to cases where the design matrix is rank deficient or has more columns than rows, including the important high-dimensional setting where p>N.

Contribution

It generalizes previous results by removing the full rank assumption on the design matrix, allowing for improper priors and high-dimensional models with p>N.

Findings

Gibbs samplers are nearly always geometrically ergodic under broader conditions.

The analysis accommodates rank-deficient design matrices.

The results apply to high-dimensional models with more parameters than observations.

Abstract

Exploration of the intractable posterior distributions associated with Bayesian versions of the general linear mixed model is often performed using Markov chain Monte Carlo. In particular, if a conditionally conjugate prior is used, then there is a simple two-block Gibbs sampler available. Rom\'{a}n and Hobert [Linear Algebra Appl. 473 (2015) 54-77] showed that, when the priors are proper and the $X$ matrix has full column rank, the Markov chains underlying these Gibbs samplers are nearly always geometrically ergodic. In this paper, Rom\'{a}n and Hobert's (2015) result is extended by allowing improper priors on the variance components, and, more importantly, by removing all assumptions on the $X$ matrix. So, not only is $X$ allowed to be (column) rank deficient, which provides additional flexibility in parameterizing the fixed effects, it is also allowed to have more columns than rows, which is necessary in the increasingly important situation where $p>N$. The full rank assumption on $X$ is at the heart of Rom\'{a}n and Hobert's (2015) proof. Consequently, the extension to unrestricted $X$ requires a substantially different analysis.