Correlated pseudo-marginal Metropolis-Hastings using quasi-Newton proposals

TL;DR

This paper introduces a novel correlated pseudo-marginal Metropolis-Hastings algorithm that employs quasi-Newton proposals, leveraging gradient and curvature information to improve sampling efficiency in high-dimensional, complex target distributions.

Contribution

The paper extends pmMH by integrating quasi-Newton methods for proposal construction, enabling more efficient sampling using gradient and curvature information from multiple past iterations.

Findings

qN proposals outperform other Hessian-based proposals.

The method improves sampling efficiency in high-dimensional problems.

Demonstrated on several benchmark problems.

Abstract

Pseudo-marginal Metropolis-Hastings (pmMH) is a versatile algorithm for sampling from target distributions which are not easy to evaluate point-wise. However, pmMH requires good proposal distributions to sample efficiently from the target, which can be problematic to construct in practice. This is especially a problem for high-dimensional targets when the standard random-walk proposal is inefficient. We extend pmMH to allow for constructing the proposal based on information from multiple past iterations. As a consequence, quasi-Newton (qN) methods can be employed to form proposals which utilize gradient information to guide the Markov chain to areas of high probability and to construct approximations of the local curvature to scale step sizes. The proposed method is demonstrated on several problems which indicate that qN proposals can perform better than other common Hessian-based proposals.