About the probability distribution of a quantity with given mean and variance

TL;DR

This paper discusses how to determine the probability distribution of a quantity with known mean, variance, and range, advocating for the use of minimum Kullback entropy over maximum entropy in certain cases.

Contribution

It introduces the use of the minimum Kullback entropy principle for deriving distributions when the range of a variable is known, extending the GUM-S1 recommendations.

Findings

Maximum entropy yields Gaussian distributions for known mean and variance.

Minimum Kullback entropy leads to exponential distributions when range is known.

The deviation between MaxEnt and mKE distributions is quantified, justifying GUM-S1 use.

Abstract

Supplement 1 to GUM (GUM-S1) recommends the use of maximum entropy principle (MaxEnt) in determining the probability distribution of a quantity having specified properties, e.g., specified central moments. When we only know the mean value and the variance of a variable, GUM-S1 prescribes a Gaussian probability distribution for that variable. When further information is available, in the form of a finite interval in which the variable is known to lie, we indicate how the distribution for the variable in this case can be obtained. A Gaussian distribution should only be used in this case when the standard deviation is small compared to the range of variation (the length of the interval). In general, when the interval is finite, the parameters of the distribution should be evaluated numerically, as suggested by I. Lira, Metrologia, 46 L27 (2009). Here we note that the knowledge of the range of variation is equivalent to a bias of the distribution toward a flat distribution in that range, and the principle of minimum Kullback entropy (mKE) should be used in the derivation of the probability distribution rather than the MaxEnt, thus leading to an exponential distribution with non Gaussian features. Furthermore, up to evaluating the distribution negentropy, we quantify the deviation of mKE distributions from MaxEnt ones and, thus, we rigorously justify the use of GUM-S1 recommendation also if we have further information on the range of variation of a quantity, namely, provided that its standard uncertainty is sufficiently small compared to the range.