A Multi-armed Bandit MCMC, with applications in sampling from doubly intractable posterior

TL;DR

This paper introduces Multi-armed Bandit MCMC, a novel algorithm that adaptively selects between multiple acceptance ratios to improve sampling efficiency in doubly intractable Bayesian posterior problems.

Contribution

It proposes a new MCMC method that dynamically chooses acceptance ratios, enhancing sampling performance for complex distributions.

Findings

MABMC achieves higher average acceptance probabilities.

It effectively combines existing solutions like Pseudo-marginal and Exchange algorithms.

Demonstrates improved sampling efficiency in doubly intractable problems.

Abstract

Markov chain Monte Carlo (MCMC) algorithms are widely used to sample from complicated distributions, especially to sample from the posterior distribution in Bayesian inference. However, MCMC is not directly applicable when facing the doubly intractable problem. In this paper, we discussed and compared two existing solutions -- Pseudo-marginal Monte Carlo and Exchange Algorithm. This paper also proposes a novel algorithm: Multi-armed Bandit MCMC (MABMC), which chooses between two (or more) randomized acceptance ratios in each step. MABMC could be applied directly to incorporate Pseudo-marginal Monte Carlo and Exchange algorithm, with higher average acceptance probability.