Comment on "Optimal prior for Bayesian inference in a constrained parameter space" by S. Hannestad and T. Tram, arXiv:1710.08899

TL;DR

This paper clarifies that the Jeffreys prior remains unchanged when applied to constrained parameter spaces, countering previous claims that it differs from the unconstrained case.

Contribution

It provides a correction to prior assertions, demonstrating that the Jeffreys prior is invariant under parameter constraints.

Findings

Jeffreys prior is the same for constrained and unconstrained spaces

Counter to previous claims, the prior remains unchanged under constraints

Clarifies the correct application of Jeffreys prior in constrained Bayesian inference

Abstract

The Jeffreys prior for a constrained part of a parameter space is the same as that for the unconstrained space, contrary to the assertions of Hannestad and Tram.

Full text

Comment on “Optimal prior for Bayesian inference in a

constrained parameter space” by S. Hannestad and T. Tram, arXiv:1710.08899

Robert D. Cousins

Dept. of Physics and Astronomy, University of California, Los Angeles, California 90095 USA cousins@physics.ucla.edu

(February 20, 2019)

Abstract: The Jeffreys prior for a constrained part of a parameter space is the same as that for the unconstrained space, contrary to the assertions of Hannestad and Tram.

An arXiv post by Hannestad and Tram [1] states,

Under the assumption that the likelihood is a Gaussian distribution, the Jeffreys prior is a constant, i.e. flat. However, if one parameter is constrained by physical considerations, the Gaussian approximation fails and the flat prior is no longer the Jeffreys prior…In this paper we compute the correct Jeffreys prior for a multivariate normal distribution constrained in one dimension…

When this was pointed out to me last week, I thought this odd since I have long been teaching (including recently in Section 5.3 of Ref. [2]), that such a constraint does not modify the Jeffreys prior, based on my memory of Jim Berger’s book [3]. Since this specific result of Ref. [1] is being cited without criticism [4, 5, 6, 7, 8], it is worth opening Berger’s book and finding on page 89:

One important feature of the Jeffreys noninformative prior is that it is not affected by a restriction on the parameter space. Thus, if it is known in Example 5 that $\theta>0$, the Jeffreys noninformative prior is still $\pi(\theta)=1$ (on $(0,\infty)$, of course). This is important, because one of the situations in which noninformative priors prove to be extremely useful is when dealing with restricted parameter spaces (see Chapter 4). In such situations we will, therefore, simply assume that the noninformative prior is that which is inherited from the unrestricted parameter space.

(Example 5 is for a general location parameter, which includes the Gaussian case of Ref. [1].) As the Jeffreys prior is problematic with more than one parameter, “reference priors” were developed by Bernardo and Berger. According to Ref. [9], in general they share this important feature:

Another common type of partial information is constraints on the parameter space. This is typically easily handled, however, in that reference priors for a constrained space are almost always just the unconstrained reference prior times the indicator function on the constrained space.

It appears that Hannestad and Tram have fallen into a trap that I discuss in Section 6.9 of Ref. [2], namely the “Famous confusion re Gaussian $p(x|\mu)$ where $\mu$ is mass $\geq 0$.” That is, they view the sampled value $x$ (called $q$ in their paper) as an estimate of $\mu$ and thus consider only $x\geq 0$. In truth, there is nothing anomalous about a sampled $x<0$, and the computation of the expectation value over $x$ should not be restricted to $x\geq 0$. Once this is understood (that restrictions in the parameter space do not restrict the sample space just because one may like to think of $x$ as the estimate or “measured value” of the parameter), the quoted passage from Berger’s book becomes obvious. If one still has trouble distinguishing sampled value of $x$ from point estimate of $\mu$, Section 6.4 of Ref. [2] has an instructive example. As noted at the end of Section 6.8, the confusion is also avoided if $x$ and $\mu$ have different units or dimensions.

In any case, it would seem that this result of Hannestad and Tram is contradicted by well-established professional statistics literature.

Acknowledgments

I think Maurizio Pierini for pointing me to Ref. [1]. This work was partially supported by the U.S. Department of Energy under Award Number DE–SC0009937.