Accelerating Metropolis-within-Gibbs sampler with localized computations of differential equations

TL;DR

This paper presents a novel local computation approach to accelerate Metropolis-within-Gibbs sampling for high-dimensional inverse problems governed by stochastic differential equations, reducing computational complexity from quadratic to linear.

Contribution

It introduces a local domain approximation method that significantly speeds up MwG sampling in high-dimensional SDE models by controlling approximation errors through local domain size.

Findings

Achieves $O(n)$ computation cost for inverse SDE problems

Demonstrates effectiveness with Lorenz 96 and linear stochastic flow models

Provides theoretical error bounds related to local domain size

Abstract

Inverse problem is ubiquitous in science and engineering, and Bayesian methodologies are often used to infer the underlying parameters. For high dimensional temporal-spatial models, classical Markov chain Monte Carlo (MCMC) methods are often slow to converge, and it is necessary to apply Metropolis-within-Gibbs (MwG) sampling on parameter blocks. However, the computation cost of each MwG iteration is typically $O(n^2)$, where $n$ is the model dimension. This can be too expensive in practice. This paper introduces a new reduced computation method to bring down the computation cost to $O(n)$, for the inverse initial value problem of a stochastic differential equation (SDE) with local interactions. The key observation is that each MwG proposal is only different from the original iterate at one parameter block, and this difference will only propagate within a local domain in the SDE computations. Therefore we can approximate the global SDE computation with a surrogate updated only within the local domain for reduced computation cost. Both theoretically and numerically, we show that the approximation errors can be controlled by the local domain size. We discuss how to implement the local computation scheme using Euler--Maruyama and 4th order Runge--Kutta methods. We numerically demonstrate the performance of the proposed method with the Lorenz 96 model and a linear stochastic flow model.