The Importance Markov Chain

TL;DR

The paper introduces the Importance Markov chain, a new algorithm that combines rejection and importance sampling via a tuning parameter, enabling efficient exploration of complex distributions.

Contribution

It proposes a novel Markov chain method that bridges rejection and importance sampling, with theoretical guarantees and practical ease of implementation.

Findings

Establishes Law of Large Numbers and CLT for the algorithm

Proves geometric ergodicity under mild conditions

Demonstrates efficient exploration of multimodal distributions

Abstract

The Importance Markov chain is a novel algorithm bridging the gap between rejection sampling and importance sampling, moving from one to the other through a tuning parameter. Based on a modified sample of an instrumental Markov chain targeting an instrumental distribution (typically via a MCMC kernel), the Importance Markov chain produces an extended Markov chain where the marginal distribution of the first component converges to the target distribution. For example, when targeting a multimodal distribution, the instrumental distribution can be chosen as a tempered version of the target which allows the algorithm to explore its modes more efficiently. We obtain a Law of Large Numbers and a Central Limit Theorem as well as geometric ergodicity for this extended kernel under mild assumptions on the instrumental kernel. Computationally, the algorithm is easy to implement and preexisting libraries can be used to sample from the instrumental distribution.