Comparison of hit-and-run, slice sampling and random walk Metropolis

TL;DR

This paper compares the efficiency of hit-and-run, slice sampling, and random walk Metropolis algorithms for sampling from unnormalized distributions, establishing a hierarchy of their effectiveness using covariance ordering.

Contribution

The authors introduce a general framework using covariance ordering to compare Markov chain sampling algorithms, revealing the relative efficiencies of popular methods.

Findings

Hit-and-run and simple slice sampler outperform hybrid slice sampler.

Hybrid slice sampler is more efficient than lazy random walk Metropolis.

Efficiency order: hit-and-run and slice sampler > hybrid slice sampler > random walk Metropolis.

Abstract

Different Markov chains can be used for approximate sampling of a distribution given by an unnormalized density function with respect to the Lebesgue measure. The hit-and-run, (hybrid) slice sampler and random walk Metropolis algorithm are popular tools to simulate such Markov chains. We develop a general approach to compare the efficiency of these sampling procedures by the use of a partial ordering of their Markov operators, the covariance ordering. In particular, we show that the hit-and-run and the simple slice sampler are more efficient than a hybrid slice sampler based on hit-and-run which, itself, is more efficient than a (lazy) random walk Metropolis algorithm.