Some Simple Formulas for Posterior Convergence Rates

TL;DR

This paper presents simple, explicit formulas linking posterior convergence rates to prior properties, facilitating easier analysis and optimization of Bayesian models without strong assumptions.

Contribution

It introduces straightforward relations connecting convergence rates to prior divergence and complexity, applicable to model averaging and adaptive performance optimization.

Findings

Derived explicit formulas for posterior convergence rates.

Applied formulas to model averaging, obtaining oracle inequalities.

Enabled adaptive performance optimization without knowing the true model.

Abstract

We derive some simple relations that demonstrate how the posterior convergence rate is related to two driving factors: a "penalized divergence" of the prior, which measures the ability of the prior distribution to propose a nonnegligible set of working models to approximate the true model and a "norm complexity" of the prior, which measures the complexity of the prior support, weighted by the prior probability masses. These formulas are explicit and involve no essential assumptions and are easy to apply. We apply this approach to the case with model averaging and derive some useful oracle inequalities that can optimize the performance adaptively without knowing the true model.