Confronting classical and Bayesian confidence limits to examples

TL;DR

This paper compares classical and Bayesian confidence limits through examples, highlighting issues with classical methods and proposing a likelihood-based approach aligned with high energy physics practices.

Contribution

It introduces a likelihood-based error limit definition that addresses limitations of classical and Bayesian methods, promoting a more consistent approach.

Findings

Classical confidence limits violate the likelihood principle.

Classical methods can produce physically inconsistent results.

Likelihood-based error limits are more coherent and general.

Abstract

Classical confidence limits are compared to Bayesian error bounds by studying relevant examples. The performance of the two methods is investigated relative to the properties coherence, precision, bias, universality, simplicity. A proposal to define error limits in various cases is derived from the comparison. It is based on the likelihood function only and follows in most cases the general practice in high energy physics. Classical methods are discarded because they violate the likelihood principle, they can produce physically inconsistent results, suffer from a lack of precision and generality. Also the extreme Bayesian approach with arbitrary choice of the prior probability density or priors deduced from scaling laws is rejected.