The Art of Probability Assignment

TL;DR

This paper discusses probability assignment for physical observables with limited data, proposing methods that minimize sensitivity to priors, using Fisher information and Renyi distance, to improve inference accuracy.

Contribution

It introduces a novel approach to probability assignment that minimizes prior sensitivity by employing Fisher information and Renyi distance, especially under scarce data conditions.

Findings

Probability assignments become insensitive to priors with abundant data.

Minimizing Fisher information leads to optimal probability distributions with limited data.

Renyi distance-based method effectively handles discrete case probability shifts.

Abstract

The problem of assigning probabilities when little is known is analized in the case where the quanities of interest are physical observables, i.e. can be measured and their values expressed by numbers. It is pointed out that the assignment of probabilities based on observation is a process of inference, involving the use of Bayes' theorem and the choice of a probability prior. When a lot of data is available, the resulting probability are remarkably insensitive to the form of the prior. In the oposite case of scarse data, it is suggested that the probabilities are assigned such that they are the least sensitive to specific variations of the probability prior. In the continuous case this results in a probability assignment rule wich calls for minimizing the Fisher information subject to constraints reflecting all available information. In the discrete case, the corresponding quantity to be minimized turns out to be a Renyi distance between the original and the shifted distribution.