Stability of the Gibbs Sampler for Bayesian Hierarchical Models

TL;DR

This paper analyzes the convergence behavior of the Gibbs sampler in Bayesian hierarchical models, revealing how error distribution tails and parametrization influence convergence rates, with applications to latent Gaussian process models.

Contribution

It provides a theoretical framework characterizing Gibbs sampler convergence in hierarchical models with arbitrary symmetric errors, including uniform, geometric, and sub-geometric cases.

Findings

Convergence depends on error tail behavior and parametrization.

The framework applies to latent Gaussian process models.

Guidelines for analyzing complex hierarchical models.

Abstract

We characterise the convergence of the Gibbs sampler which samples from the joint posterior distribution of parameters and missing data in hierarchical linear models with arbitrary symmetric error distributions. We show that the convergence can be uniform, geometric or sub-geometric depending on the relative tail behaviour of the error distributions, and on the parametrisation chosen. Our theory is applied to characterise the convergence of the Gibbs sampler on latent Gaussian process models. We indicate how the theoretical framework we introduce will be useful in analyzing more complex models.