MALA-within-Gibbs samplers for high-dimensional distributions with sparse conditional structure

TL;DR

This paper analyzes the MALA-within-Gibbs sampler, showing its efficiency in high-dimensional problems with sparse conditional structure, and demonstrates its practical utility in Bayesian inference with theoretical and numerical insights.

Contribution

It establishes that the acceptance ratio, step size, and convergence rate of MALA-within-Gibbs are independent of dimension under certain sparsity and log-concavity conditions, providing theoretical and practical guidance.

Findings

Acceptance ratio and step size are dimension-independent under sparsity.

Convergence rate is independent of dimension for block-wise log-concave targets.

Numerical examples illustrate the importance of correctly partitioning the state space.

Abstract

Markov chain Monte Carlo (MCMC) samplers are numerical methods for drawing samples from a given target probability distribution. We discuss one particular MCMC sampler, the MALA-within-Gibbs sampler, from the theoretical and practical perspectives. We first show that the acceptance ratio and step size of this sampler are independent of the overall problem dimension when (i) the target distribution has sparse conditional structure, and (ii) this structure is reflected in the partial updating strategy of MALA-within-Gibbs. If, in addition, the target density is block-wise log-concave, then the sampler's convergence rate is independent of dimension. From a practical perspective, we expect that MALA-within-Gibbs is useful for solving high-dimensional Bayesian inference problems where the posterior exhibits sparse conditional structure at least approximately. In this context, a partitioning of the state that correctly reflects the sparse conditional structure must be found, and we illustrate this process in two numerical examples. We also discuss trade-offs between the block size used for partial updating and computational requirements that may increase with the number of blocks.