Geometric convergence of elliptical slice sampling

TL;DR

This paper proves that elliptical slice sampling, a popular Bayesian sampling method, converges geometrically under certain conditions, and demonstrates its effectiveness and dimension-independence in various scenarios.

Contribution

It establishes geometric ergodicity of elliptical slice sampling under weak regularity assumptions, providing theoretical convergence guarantees for the method.

Findings

Markov chain is geometrically ergodic under weak conditions

Elliptical slice sampling performs dimension-independently in experiments

Numerical results support theoretical convergence in Gaussian and multi-modal cases

Abstract

For Bayesian learning, given likelihood function and Gaussian prior, the elliptical slice sampler, introduced by Murray, Adams and MacKay 2010, provides a tool for the construction of a Markov chain for approximate sampling of the underlying posterior distribution. Besides of its wide applicability and simplicity its main feature is that no tuning is necessary. Under weak regularity assumptions on the posterior density we show that the corresponding Markov chain is geometrically ergodic and therefore yield qualitative convergence guarantees. We illustrate our result for Gaussian posteriors as they appear in Gaussian process regression, as well as in a setting of a multi-modal distribution. Remarkably, our numerical experiments indicate a dimension-independent performance of elliptical slice sampling even in situations where our ergodicity result does not apply.