Qualitative Robustness in Bayesian Inference

TL;DR

This paper investigates the robustness of Bayesian inference under numerical approximations, focusing on how errors in prior and data affect the convergence and stability of posterior distributions in large data limits.

Contribution

It provides a theoretical analysis of the conditions under which Bayesian inference remains robust despite approximation errors in prior and data distributions.

Findings

Identifies norms in which PDE solutions should be approximated for robustness

Analyzes the impact of prior and data perturbations on posterior distribution stability

Examines the asymptotic behavior of posterior distributions as data size grows

Abstract

The practical implementation of Bayesian inference requires numerical approximation when closed-form expressions are not available. What types of accuracy (convergence) of the numerical approximations guarantee robustness and what types do not? In particular, is the recursive application of Bayes' rule robust when subsequent data or posteriors are approximated? When the prior is the push forward of a distribution by the map induced by the solution of a PDE, in which norm should that solution be approximated? Motivated by such questions, we investigate the sensitivity of the distribution of posterior distributions (i.e. posterior distribution-valued random variables, randomized through the data) with respect to perturbations of the prior and data generating distributions in the limit when the number of data points grows towards infinity.