Is Bayes Posterior just Quick and Dirty Confidence?

TL;DR

This paper compares Bayesian posterior and confidence distributions, showing that Bayesian methods can be misleading in non-linear models, and that Bayesian integration of weighted likelihood approximates confidence only under linearity.

Contribution

It critically examines the validity of Bayesian posterior statements versus confidence statements, highlighting the importance of model linearity for accurate Bayesian inference.

Findings

Bayesian posterior accuracy depends on model linearity.

Departure from linearity can cause Bayesian methods to be misleading.

Bayesian weighted likelihood approximates confidence in linear models.

Abstract

Bayes [Philos. Trans. R. Soc. Lond. 53 (1763) 370--418; 54 296--325] introduced the observed likelihood function to statistical inference and provided a weight function to calibrate the parameter; he also introduced a confidence distribution on the parameter space but did not provide present justifications. Of course the names likelihood and confidence did not appear until much later: Fisher [Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 222 (1922) 309--368] for likelihood and Neyman [Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 237 (1937) 333--380] for confidence. Lindley [J. Roy. Statist. Soc. Ser. B 20 (1958) 102--107] showed that the Bayes and the confidence results were different when the model was not location. This paper examines the occurrence of true statements from the Bayes approach and from the confidence approach, and shows that the proportion of true statements in the Bayes case depends critically on the presence of linearity in the model; and with departure from this linearity the Bayes approach can be a poor approximation and be seriously misleading. Bayesian integration of weighted likelihood thus provides a first-order linear approximation to confidence, but without linearity can give substantially incorrect results.