The Physical Significance of Confidence Intervals

TL;DR

This paper introduces new statistical quantities to assess the physical significance and reliability of confidence intervals in both Frequentist and Bayesian frameworks, enhancing interpretation of experimental results.

Contribution

It defines novel metrics like exclusion potential, Pull, detection functions, and a new sensitivity measure to better evaluate experimental outcomes and their significance.

Findings

Introduces the concept of exclusion potential and its standard deviation.

Defines the Pull as a measure of reliability for null results.

Proposes new detection functions and a refined sensitivity measure.

Abstract

We define some appropriate statistical quantities that indicate the physical significance (reliability) of confidence intervals in the framework of both Frequentist and Bayesian statistical theories. We consider the expectation value of the upper limit in the absence of a signal (that we propose to call "exclusion potential", instead of "sensitivity" as done by Feldman and Cousins) and its standard deviation, we define the "Pull" of a null result, expressing the reliability of an experimental upper limit, and the "upper and lower detection functions", that give information on the possible outcome of an experiment if there is a signal. We also give a new appropriate definition of "sensitivity", that quantifies the capability of an experiment to reveal the signal that is searched for at the given confidence level.