On the Theoretical Properties of the Exchange Algorithm

TL;DR

This paper investigates the theoretical properties of the exchange algorithm, including convergence rates and asymptotic behavior, providing conditions for geometric ergodicity and establishing a central limit theorem.

Contribution

It offers the first comprehensive theoretical analysis of the exchange algorithm, including convergence conditions and applicability to various models.

Findings

Provides necessary and sufficient conditions for geometric ergodicity.

Establishes a central limit theorem for the exchange algorithm.

Demonstrates applicability to multiple practical models.

Abstract

The exchange algorithm is one of the most popular extensions of the Metropolis--Hastings algorithm to sample from doubly-intractable distributions. However, the theoretical exploration of the exchange algorithm is very limited. For example, natural questions like `Does exchange algorithm converge at a geometric rate?' or `Does the exchange algorithm admit a Central Limit Theorem?' have not been answered yet. In this paper, we study the theoretical properties of the exchange algorithm, in terms of asymptotic variance and convergence speed. We compare the exchange algorithm with the original Metropolis--Hastings algorithm and provide both necessary and sufficient conditions for the geometric ergodicity of the exchange algorithm. Moreover, we prove that our results can be applied to various practical applications such as location models, Gaussian models, Poisson models, and a large class of exponential families, which includes most of the practical applications of the exchange algorithm. A central limit theorem for the exchange algorithm is also established. Our results justify the theoretical usefulness of the exchange algorithm.