The Power of Confidence Intervals

TL;DR

This paper explores the relationship between the power of confidence intervals and their physical significance, emphasizing how different methods vary in the reliability of their upper limits for bounded parameters.

Contribution

It establishes a connection between the power of confidence intervals and their physical significance, comparing various methods for bounded parameters.

Findings

Methods with high power against larger alternatives yield more physically significant upper limits.

The Maximum Likelihood Estimator method provides the most physically significant upper limits among the methods considered.

The Unified Approach has the least physically significant upper limits among the methods analyzed.

Abstract

We consider the power to reject false values of the parameter in Frequentist methods for the calculation of confidence intervals. We connect the power with the physical significance (reliability) of confidence intervals for a parameter bounded to be non-negative. We show that the confidence intervals (upper limits) obtained with a (biased) method that near the boundary has large power in testing the parameter against larger alternatives and small power in testing the parameter against smaller alternatives are physically more significant. Considering the recently proposed methods with correct coverage, we show that the physical significance of upper limits is smallest in the Unified Approach and highest in the Maximum Likelihood Estimator method. We illustrate our arguments in the specific cases of a bounded Gaussian distribution and a Poisson distribution with known background.