Data-driven priors and their posterior concentration rates

TL;DR

This paper proposes a data-driven approach to constructing priors in high-dimensional Bayesian problems, ensuring desirable posterior concentration rates while allowing flexible prior shapes.

Contribution

It introduces a general strategy for empirical prior construction that guarantees posterior concentration rates and reduces sensitivity to prior shape.

Findings

Empirical priors can achieve optimal concentration rates.

Posterior properties become less sensitive to prior shape.

Applicable to density estimation, regression, and high-dimensional models.

Abstract

In high-dimensional problems, choosing a prior distribution such that the corresponding posterior has desirable practical and theoretical properties can be challenging. This begs the question: can the data be used to help choose a good prior? In this paper, we develop a general strategy for constructing a data-driven or empirical prior and sufficient conditions for the corresponding posterior distribution to achieve a certain concentration rate. The idea is that the prior should put sufficient mass on parameter values for which the likelihood is large. An interesting byproduct of this data-driven centering is that the asymptotic properties of the posterior are less sensitive to the prior shape which, in turn, allows users to work with priors of computationally convenient forms while maintaining the desired rates. General results on both adaptive and non-adaptive rates based on empirical priors are presented, along with illustrations in density estimation, nonparametric regression, and high-dimensional structured normal models.