Accelerating Metropolis-Hastings algorithms by Delayed Acceptance

TL;DR

This paper introduces a generalized Delayed Acceptance method to speed up Metropolis-Hastings algorithms by dividing the acceptance step, significantly reducing computational costs for complex target distributions.

Contribution

It proposes a universal divide-and-conquer acceleration strategy for Metropolis-Hastings, with theoretical bounds and optimization insights, applicable to large datasets.

Findings

Significant reduction in computation time demonstrated

Theoretical bounds for estimator variance established

Optimal scaling strategies identified

Abstract

MCMC algorithms such as Metropolis-Hastings algorithms are slowed down by the computation of complex target distributions as exemplified by huge datasets. We offer in this paper a useful generalisation of the Delayed Acceptance approach, devised to reduce the computational costs of such algorithms by a simple and universal divide-and-conquer strategy. The idea behind the generic acceleration is to divide the acceptance step into several parts, aiming at a major reduction in computing time that out-ranks the corresponding reduction in acceptance probability. Each of the components can be sequentially compared with a uniform variate, the first rejection signalling that the proposed value is considered no further. We develop moreover theoretical bounds for the variance of associated estimators with respect to the variance of the standard Metropolis-Hastings and detail some results on optimal scaling and general optimisation of the procedure. We illustrate those accelerating features on a series of examples