Statistical Guarantees for Data-driven Posterior Tempering

TL;DR

This paper provides theoretical insights and statistical guarantees for data-driven selection of the tempering parameter in fractional posteriors, enhancing robustness and understanding of their asymptotic behavior.

Contribution

It formalizes the asymptotic properties of power posteriors, introduces a new Laplace approximation, and identifies a critical threshold for the tempering parameter based on data.

Findings

Consistency of power posterior moments under certain conditions

Asymptotic normality of the power posterior mean

Identification of a threshold $\alpha \\asymp 1/\\sqrt{n}$ for asymptotic normality

Abstract

Posterior tempering reduces the influence of the likelihood in the calculation of the posterior by raising the likelihood to a fractional power $\alpha$. The resulting power posterior - also known as an $\alpha$-posterior or fractional posterior - has been shown to exhibit appealing properties, including robustness to model misspecification and asymptotic normality (Bernstein-von Mises theorem). However, practical recommendations for selecting the tempering parameter and statistical guarantees for the resulting power posterior remain open questions. Cross-validation-based approaches to tuning this parameter suggest interesting asymptotic regimes for the selected $\alpha$, which can either vanish or behave like a mixture distribution with a point mass at infinity and the remaining mass converging to zero. We formalize the asymptotic properties of the power posterior in these regimes. In particular, we provide sufficient conditions for (i) consistency of the power posterior moments and (ii) asymptotic normality of the power posterior mean. Our analysis required us to establish a new Laplace approximation that is interesting in its own right and is the key technical tool for showing a critical threshold $\alpha\asymp 1/\sqrt{n}$ where the asymptotic normality of the posterior mean breaks. Our results allow for the power to depend on the data in an arbitrary way.