The semi-parametric Bernstein-von Mises theorem for regression models with symmetric errors

TL;DR

This paper proves a semi-parametric Bernstein-von Mises theorem for regression models with symmetric errors, showing that Bayesian posterior distributions are asymptotically equivalent to efficient estimators, leading to optimal inference.

Contribution

It establishes the semi-parametric Bernstein-von Mises theorem for models with symmetric errors, including linear and mixed effect models, with mild prior conditions.

Findings

Posterior distributions match efficient estimators asymptotically.

Bayes estimators achieve frequentist optimality in these models.

Provides efficient estimation for linear mixed effect models.

Abstract

In a smooth semi-parametric model, the marginal posterior distribution for a finite dimensional parameter of interest is expected to be asymptotically equivalent to the sampling distribution of any efficient point-estimator. The assertion leads to asymptotic equivalence of credible and confidence sets for the parameter of interest and is known as the semi-parametric Bernstein-von Mises theorem. In recent years, it has received much attention and has been applied in many examples. We consider models in which errors with symmetric densities play a role; more specifically, it is shown that the marginal posterior distributions of regression coefficients in the linear regression and linear mixed effect models satisfy the semi-parametric Bernstein-von Mises assertion. As a consequence, Bayes estimators in these models achieve frequentist inferential optimality, as expressed e.g. through Hajek's convolution and asymptotic minimax theorems. Conditions for the prior on the space of error densities are relatively mild and well-known constructions like the Dirichlet process mixture of normal densities and random series priors constitute valid choices. Particularly, the result provides an efficient estimate of regression coefficients in the linear mixed effect model, for which no other efficient point-estimator was known previously.