Comments on the Unified approach to the construction of Classical confidence intervals

TL;DR

This paper critiques the properties of the Unified approach to confidence intervals, clarifies misconceptions about its behavior, and discusses how different interval methods exhibit similar transition to upper limits as signals weaken.

Contribution

It clarifies that the behavior of transitioning to upper limits is not unique to the Unified approach and proposes clearer terminology for the F&C limit.

Findings

The flip-flopping problem can be avoided with proper confidence level choices.

Different interval methods show similar behavior in weak signal scenarios.

The paper proposes clearer terminology for the F&C limit.

Abstract

The paper comments on properties of the so-called "Unified approach to the construction of classical confidence intervals", in which confidence intervals are computed in a Neyman construction using the likelihood ratio as ordering quantity. In particular, two of the main results of a paper by Feldman and Cousins (F&C) are discussed. It is shown that in the case of central intervals the so-called flip-flopping problem, occuring in the specific scenario where the experimenter decides to quote a standard upper limit or a confidence interval depending on the measurement, can be easily avoided by choosing appropriate confidence levels for the standard upper limits and confidence intervals. In the F&C paper "upper limit" is defined as the upper edge of a confidence interval, whose lower edge coincides with the physical limit. With this definition of upper limit (F&C limit), in an approach which uses the likelihood ratio as ordering quantity, two-sided confidence intervals automatically change over to "upper limits" as the signal becomes weaker (Unified approach). In the present paper it is pointed out that this behaviour is not a special property of this approach, because approaches with other ordering principles, like central intervals, symmetric intervals or highest-probability intervals, exhibit the same behaviour. The Unified approach is presented in the F&C paper as a solution to the flip-flopping problem. This might suggest that the F&C limit is a standard upper limit. In order to exclude any misunderstanding, it is proposed in the present paper to call the F&C limit "upper edge of the confidence interval", even if its lower edge coincides with the physical limit.