Gibbs Sampling using Anti-correlation Gaussian Data Augmentation, with Applications to L1-ball-type Models

TL;DR

This paper introduces a novel anti-correlation Gaussian data augmentation for L1-ball priors, enabling faster block Gibbs sampling with guaranteed geometric ergodicity and broad applicability to latent Gaussian models.

Contribution

It proposes a new data augmentation technique that simplifies posterior computation for L1-ball models, significantly improving sampling efficiency and mixing.

Findings

The new Gibbs sampler has lower computational cost per iteration.

It demonstrates rapid mixing compared to existing algorithms.

The method is extendable to complex latent Gaussian models.

Abstract

L1-ball-type priors are a recent generalization of the spike-and-slab priors. By transforming a continuous precursor distribution to the L1-ball boundary, it induces exact zeros with positive prior and posterior probabilities. With great flexibility in choosing the precursor and threshold distributions, we can easily specify models under structured sparsity, such as those with dependent probability for zeros and smoothness among the non-zeros. Motivated to significantly accelerate the posterior computation, we propose a new data augmentation that leads to a fast block Gibbs sampling algorithm. The latent variable, named ``anti-correlation Gaussian'', cancels out the quadratic exponent term in the latent Gaussian distribution, making the parameters of interest conditionally independent so that they can be updated in a block. Compared to existing algorithms such as the No-U-Turn sampler, the new blocked Gibbs sampler has a very low computing cost per iteration and shows rapid mixing of Markov chains. We establish the geometric ergodicity guarantee of the algorithm in linear models. Further, we show useful extensions of our algorithm for posterior estimation of general latent Gaussian models, such as those involving multivariate truncated Gaussian or latent Gaussian process.