Parallel local approximation MCMC for expensive models

TL;DR

This paper introduces a parallel local approximation MCMC method that reduces computational costs for expensive models by collaboratively constructing shared approximations, enabling faster Bayesian inference in complex scientific problems.

Contribution

It extends previous local approximation MCMC by enabling parallel chains to collaboratively build a shared posterior approximation while maintaining ergodicity.

Findings

Significant reduction in inference time for hydrology and glaciology models

Parallel chains effectively share approximation information

Method maintains asymptotic correctness of the Markov chain

Abstract

Performing Bayesian inference via Markov chain Monte Carlo (MCMC) can be exceedingly expensive when posterior evaluations invoke the evaluation of a computationally expensive model, such as a system of partial differential equations. In recent work [Conrad et al. JASA 2016, arXiv:1402.1694], we described a framework for constructing and refining local approximations of such models during an MCMC simulation. These posterior--adapted approximations harness regularity of the model to reduce the computational cost of inference while preserving asymptotic exactness of the Markov chain. Here we describe two extensions of that work. First, we prove that samplers running in parallel can collaboratively construct a shared posterior approximation while ensuring ergodicity of each associated chain, providing a novel opportunity for exploiting parallel computation in MCMC. Second, focusing on the Metropolis--adjusted Langevin algorithm, we describe how a proposal distribution can successfully employ gradients and other relevant information extracted from the approximation. We investigate the practical performance of our strategies using two challenging inference problems, the first in subsurface hydrology and the second in glaciology. Using local approximations constructed via parallel chains, we successfully reduce the run time needed to characterize the posterior distributions in these problems from days to hours and from months to days, respectively, dramatically improving the tractability of Bayesian inference.