The Bayesian Analysis of Complex, High-Dimensional Models: Can It Be CODA?

TL;DR

This paper examines the effectiveness of Bayesian methods in high-dimensional models, showing that intuitive priors often underperform compared to simple frequentist estimators, but with potential for optimal rate construction.

Contribution

It provides a strong consistency theorem for Bayesian estimators and discusses how to design priors that achieve minimax rates in complex models.

Findings

Intuitive priors can lead to poor estimators in high-dimensional settings.

A strong version of Doob's consistency theorem links estimator efficiency to posterior consistency.

Constructing priors for both global and local minimax rates is theoretically possible.

Abstract

We consider the Bayesian analysis of a few complex, high-dimensional models and show that intuitive priors, which are not tailored to the fine details of the model and the estimated parameters, produce estimators which perform poorly in situations in which good, simple frequentist estimators exist. The models we consider are: stratified sampling, the partial linear model, linear and quadratic functionals of white noise and estimation with stopping times. We present a strong version of Doob's consistency theorem which demonstrates that the existence of a uniformly $\sqrt{n}$-consistent estimator ensures that the Bayes posterior is $\sqrt{n}$-consistent for values of the parameter in subsets of prior probability 1. We also demonstrate that it is, at least, in principle, possible to construct Bayes priors giving both global and local minimax rates, using a suitable combination of loss functions. We argue that there is no contradiction in these apparently conflicting findings.