On Unbiased Estimation for Discretized Models

TL;DR

This paper develops novel unbiased Monte Carlo estimators for expectations in discretized probabilistic models, enabling scalable, parallelizable inference with finite variance and cost.

Contribution

It introduces a new adaptation of randomization and Markov simulation techniques to construct unbiased estimators for discretized models, ensuring finite variance and cost.

Findings

Estimators achieve unbiased inference at canonical complexity rate.

Estimators can be generated independently, enabling parallel computation.

Applied to Bayesian inference with simulated and real data.

Abstract

In this article, we consider computing expectations w.r.t. probability measures which are subject to discretization error. Examples include partially observed diffusion processes or inverse problems, where one may have to discretize time and/or space, in order to practically work with the probability of interest. Given access only to these discretizations, we consider the construction of unbiased Monte Carlo estimators of expectations w.r.t. such target probability distributions. It is shown how to obtain such estimators using a novel adaptation of randomization schemes and Markov simulation methods. Under appropriate assumptions, these estimators possess finite variance and finite expected cost. There are two important consequences of this approach: (i) unbiased inference is achieved at the canonical complexity rate, and (ii) the resulting estimators can be generated independently, thereby allowing strong scaling to arbitrarily many parallel processors. Several algorithms are presented, and applied to some examples of Bayesian inference problems, with both simulated and real observed data.