Gradient Scan Gibbs Sampler: an efficient algorithm for high-dimensional Gaussian distributions

TL;DR

This paper introduces an efficient Gibbs sampling algorithm for high-dimensional Gaussian distributions that avoids costly sampling steps by using directional excursions, with proven convergence and applications in inverse problems.

Contribution

It proposes a novel algorithm that improves sampling efficiency in high-dimensional Gaussian models by avoiding direct high-dimensional sampling, with theoretical convergence guarantees.

Findings

Algorithm converges to the target distribution.

Effective in inverse problems and hierarchical Bayesian models.

Applicable to semi-blind and non-Gaussian methods.

Abstract

This paper deals with Gibbs samplers that include high dimensional conditional Gaussian distributions. It proposes an efficient algorithm that avoids the high dimensional Gaussian sampling and relies on a random excursion along a small set of directions. The algorithm is proved to converge, i.e. the drawn samples are asymptotically distributed according to the target distribution. Our main motivation is in inverse problems related to general linear observation models and their solution in a hierarchical Bayesian framework implemented through sampling algorithms. It finds direct applications in semi-blind/unsupervised methods as well as in some non-Gaussian methods. The paper provides an illustration focused on the unsupervised estimation for super-resolution methods.