Frequentist and Bayesian Confidence Intervals

TL;DR

This paper compares frequentist and Bayesian methods for constructing confidence intervals, highlighting their differences, problems, and proposing likelihood-based error limits suitable for high energy physics applications.

Contribution

It provides a critical comparison of classical and Bayesian confidence interval methods, advocating for likelihood-based limits that adhere to the Likelihood Principle.

Findings

Classical methods violate the Likelihood Principle and can produce inconsistent results.

Extreme Bayesian approaches with arbitrary priors are rejected.

Likelihood-based error limits are recommended for high energy physics.

Abstract

Frequentist (classical) and the Bayesian approaches to the construction of confidence limits are compared. Various examples which illustrate specific problems are presented. The Likelihood Principle and the Stopping Rule Paradox are discussed. The performance of the different methods is investigated relative to the properties coherence, precision, bias, universality, simplicity. A proposal on how to define error limits in various cases are derived from the comparison. They are based on the likelihood function only and follow in most cases the general practice in high energy physics. Classical methods are not recommended because they violate the Likelihood Principle, they can produce physically inconsistent results, suffer from lack of precision and generality. Also the extreme Bayesian approach with arbitrary choice of the prior probability density or priors deduced from scaling laws is rejected.