Objective Priors: An Introduction for Frequentists

TL;DR

This paper reviews objective priors in Bayesian statistics, compares their performance, and discusses their influence on inference for small to moderate samples, highlighting Jeffreys' prior and new optimal priors under divergence criteria.

Contribution

It introduces a new optimal prior under the chi-square divergence criterion, expanding the understanding of objective priors beyond Jeffreys' rule.

Findings

Jeffreys' prior has optimal properties for large samples.

New prior is optimal under chi-square divergence criterion.

Objective priors influence inference in small to moderate samples.

Abstract

Bayesian methods are increasingly applied in these days in the theory and practice of statistics. Any Bayesian inference depends on a likelihood and a prior. Ideally one would like to elicit a prior from related sources of information or past data. However, in its absence, Bayesian methods need to rely on some "objective" or "default" priors, and the resulting posterior inference can still be quite valuable. Not surprisingly, over the years, the catalog of objective priors also has become prohibitively large, and one has to set some specific criteria for the selection of such priors. Our aim is to review some of these criteria, compare their performance, and illustrate them with some simple examples. While for very large sample sizes, it does not possibly matter what objective prior one uses, the selection of such a prior does influence inference for small or moderate samples. For regular models where asymptotic normality holds, Jeffreys' general rule prior, the positive square root of the determinant of the Fisher information matrix, enjoys many optimality properties in the absence of nuisance parameters. In the presence of nuisance parameters, however, there are many other priors which emerge as optimal depending on the criterion selected. One new feature in this article is that a prior different from Jeffreys' is shown to be optimal under the chi-square divergence criterion even in the absence of nuisance parameters. The latter is also invariant under one-to-one reparameterization.