Variance bounds and robust tuning for pseudo-marginal Metropolis--Hastings algorithms

TL;DR

This paper provides new bounds on the variance of pseudo-marginal Metropolis--Hastings algorithms, offering improved guidance for tuning and highlighting the benefits of correlated proposals for better algorithm performance.

Contribution

It introduces simple upper and lower bounds on the asymptotic variance of PMMH, challenging previous tuning advice and demonstrating how correlated proposals can improve algorithm efficiency.

Findings

New bounds clarify limitations of previous tuning rules

Correlated proposals can make previously infinite variances finite

Guidelines for better tuning and proposal strategies

Abstract

The general applicability and ease of use of the pseudo-marginal Metropolis--Hastings (PMMH) algorithm, and particle Metropolis--Hastings in particular, makes it a popular method for inference on discretely observed Markovian stochastic processes. The performance of these algorithms and, in the case of particle Metropolis--Hastings, the trade off between improved mixing through increased accuracy of the estimator and the computational cost were investigated independently in two papers, both published in 2015. Each suggested choosing the number of particles so that the variance of the logarithm of the estimator of the posterior at a fixed sensible parameter value is approximately 1. This advice has been widely and successfully adopted. We provide new, remarkably simple upper and lower bounds on the asymptotic variance of PMMH algorithms. The bounds explain how blindly following the 2015 advice can hide serious issues with the algorithm and they strongly suggest an alternative criterion. In most situations our guidelines and those from 2015 closely coincide; however, when the two differ it is safer to follow the new guidance. An extension of one of our bounds shows how the use of correlated proposals can fundamentally shift the properties of pseudo-marginal algorithms, so that asymptotic variances that were infinite under the PMMH kernel become finite.