Bayesian Update with Importance Sampling: Required Sample Size

TL;DR

This paper investigates the sample size needed for importance sampling in Bayesian inference, analyzing how factors like dimension and noise influence its effectiveness, and compares standard and optimal proposals in particle filtering.

Contribution

It develops a general theoretical framework for understanding importance sampling sample size requirements and provides practical insights through numerous examples.

Findings

Sample size depends on dimension and noise level.

Optimal proposals can reduce required sample size.

Theoretical bounds are supported by numerical examples.

Abstract

Importance sampling is used to approximate Bayes' rule in many computational approaches to Bayesian inverse problems, data assimilation and machine learning. This paper reviews and further investigates the required sample size for importance sampling in terms of the $\chi^2$-divergence between target and proposal. We develop general abstract theory and illustrate through numerous examples the roles that dimension, noise-level and other model parameters play in approximating the Bayesian update with importance sampling. Our examples also facilitate a new direct comparison of standard and optimal proposals for particle filtering.