On particle Gibbs sampling

TL;DR

This paper provides a theoretical analysis of the particle Gibbs sampler, demonstrating its uniform ergodicity, improved efficiency with ancestor resampling, and extension to lower variance resampling schemes, supported by numerical experiments.

Contribution

It offers new theoretical insights into particle Gibbs, including coupling construction, ergodicity proof, and efficiency improvements through ancestor resampling and lower variance schemes.

Findings

Particle Gibbs is uniformly ergodic with enough particles.

Ancestor resampling improves mixing and reduces asymptotic variance.

Extensions to lower variance resampling schemes enhance efficiency.

Abstract

The particle Gibbs sampler is a Markov chain Monte Carlo (MCMC) algorithm to sample from the full posterior distribution of a state-space model. It does so by executing Gibbs sampling steps on an extended target distribution defined on the space of the auxiliary variables generated by an interacting particle system. This paper makes the following contributions to the theoretical study of this algorithm. Firstly, we present a coupling construction between two particle Gibbs updates from different starting points and we show that the coupling probability may be made arbitrarily close to one by increasing the number of particles. We obtain as a direct corollary that the particle Gibbs kernel is uniformly ergodic. Secondly, we show how the inclusion of an additional Gibbs sampling step that reselects the ancestors of the particle Gibbs' extended target distribution, which is a popular approach in practice to improve mixing, does indeed yield a theoretically more efficient algorithm as measured by the asymptotic variance. Thirdly, we extend particle Gibbs to work with lower variance resampling schemes. A detailed numerical study is provided to demonstrate the efficiency of particle Gibbs and the proposed variants.