Continuous Herded Gibbs Sampling

TL;DR

This paper introduces a continuous herded Gibbs sampler that combines kernel herding with Gibbs sampling, enabling efficient deterministic sampling from high-dimensional continuous distributions with reduced computation time.

Contribution

The paper presents a novel continuous herded Gibbs sampling algorithm that efficiently combines kernel herding and Gibbs sampling for high-dimensional densities.

Findings

L2 error decreases similarly to kernel herding

Computation time is significantly lower, linear in dimensions

Effective for Gaussian mixture densities

Abstract

Herding is a technique to sequentially generate deterministic samples from a probability distribution. In this work, we propose a continuous herded Gibbs sampler that combines kernel herding on continuous densities with the Gibbs sampling idea. Our algorithm allows for deterministically sampling from high-dimensional multivariate probability densities, without directly sampling from the joint density. Experiments with Gaussian mixture densities indicate that the L2 error decreases similarly to kernel herding, while the computation time is significantly lower, i.e., linear in the number of dimensions.