Priors for Reducing Asymptotic Bias of the Posterior Mean

TL;DR

This paper demonstrates that selecting an appropriate prior, specifically the squared Jeffreys prior in regular models, can eliminate the first-order asymptotic bias of the posterior mean, improving Bayesian inference accuracy.

Contribution

It introduces a prior choice that removes the first-order asymptotic bias of the posterior mean in regular models, linking it to Jeffreys and moment matching priors.

Findings

Squared Jeffreys prior removes first-order bias in exponential families.

The prior reduces bias in linear and logistic regression models.

Connections established between bias-reducing priors and moment matching priors.

Abstract

It is shown that the first-order term of the asymptotic bias of the posterior mean is removed by a suitable choice of a prior density. In regular statistical models including exponential families, and linear and logistic regression models, such a prior is given by the squared Jeffreys prior. We also explain the relationship between the proposed prior distribution, the moment matching prior, and the prior distribution that reduces the bias term of the posterior mode.