Scalable Bernoulli factories for Bayesian inference with intractable likelihoods

TL;DR

This paper introduces a divide-and-conquer Bernoulli factory MCMC method that achieves polynomial computational complexity for Bayesian inference with intractable likelihoods, improving scalability over traditional approaches.

Contribution

It proposes a novel divide-and-conquer Bernoulli factory MCMC algorithm with proven polynomial complexity, addressing scalability issues in intractable likelihood models.

Findings

Polynomial cost of degree 1.2 for one model

Polynomial cost of degree 1 for another model

Effective in Bayesian inference for intractable likelihoods

Abstract

Bernoulli factory MCMC algorithms implement accept-reject Markov chains without explicit computation of acceptance probabilities, and are used to target posterior distributions associated with intractable likelihood models. Intractable likelihoods naturally arise in continuous-time models and mixture distributions, or from the marginalisation of a tractable augmented model. Bernoulli factory MCMC algorithms often mix better than alternatives that target a tractable augmented posterior. However, for a likelihood that factorizes over observations, we show that their computational performance typically deteriorates exponentially with data size. To address this, we propose a simple divide-and-conquer Bernoulli factory MCMC algorithm and prove that it has polynomial complexity of degree between 1 and 2, with the exact degree depending on the existence of efficient unbiased estimators of the intractable likelihood ratio. We demonstrate the effectiveness of our approach with applications to Bayesian inference in two intractable likelihood models, and observe respective polynomial cost of degree 1.2 and 1 in the data size.