On the frequentist validity of Bayesian limits

TL;DR

This paper extends the frequentist interpretation of Bayesian posterior convergence, providing general theorems for consistency, convergence rates, and confidence set validity, applicable to complex and dependent data models.

Contribution

It generalizes Schwartz's KL condition, proving new theorems for posterior and Bayes factor consistency, and links credible sets to confidence sets without smoothness assumptions.

Findings

Posterior consistency and convergence rates established for complex models.

Bayes factors shown to be consistent in hypothesis testing.

Credible sets can be converted into valid confidence sets asymptotically.

Abstract

To the frequentist who computes posteriors, not all priors are useful asymptotically: in this paper Schwartz's 1965 Kullback-Leibler condition is generalised to enable frequentist interpretation of convergence of posterior distributions with the complex models and often dependent datasets in present-day statistical applications. We prove four simple and fully general frequentist theorems, for posterior consistency; for posterior rates of convergence; for consistency of the Bayes factor in hypothesis testing or model selection; and a theorem to obtain confidence sets from credible sets. The latter has a significant methodological consequence in frequentist uncertainty quantification: use of a suitable prior allows one to convert credible sets of a calculated, simulated or approximated posterior into asymptotically consistent confidence sets, in full generality. This extends the main inferential implication of the Bernstein-von Mises theorem to non-parametric models without smoothness conditions. Proofs require the existence of a Bayesian type of test sequence and priors giving rise to local prior predictive distributions that satisfy a weakened form of Le~Cam's contiguity with respect to the data distribution. Results are applied in a wide range of examples and counterexamples.