Two-sided confidence interval of a binomial proportion: how to choose?

TL;DR

This paper evaluates various methods for constructing binomial confidence intervals, introducing new criteria to assess their directional errors and recommending the Clopper-Pearson mid-P interval for practical use.

Contribution

It provides a comprehensive evaluation of existing binomial CI methods using novel criteria and offers practical recommendations for their application.

Findings

Wald's CI fails to control risks even with 32 successes

Likelihood ratio CI offers better risk balance than Wald's CI

Clopper-Pearson mid-P CI effectively controls one-sided errors

Abstract

Introduction: estimation of confidence intervals (CIs) of binomial proportions has been reviewed more than once but the directional interpretation, distinguishing the overestimation from the underestimation, was neglected while the sample size and theoretical proportion variances from experiment to experiment have not been formally taken in account. Herein, we define and apply new evaluation criteria, then give recommendations for the practical use of these CIs. Materials & methods: Google Scholar was used for bibliographic research. Evaluation criteria were (i) one-sided conditional errors, (ii) one-sided local average errors assuming a random theoretical proportion and (iii) expected half-widths of CIs. Results: Wald's CI did not control any of the risks, even when the expected number of successes reached 32. The likelihood ratio CI had a better balance than the logistic Wald CI. The Clopper-Pearson mid-P CI controlled well one-sided local average errors whereas the simple Clopper-Pearson CI was strictly conservative on both one-sided conditional errors. The percentile and basic bootstrap CIs had the same bias order as Wald's CI whereas the studentized CIs and BCa, modified for discrete bootstrap distributions, were less biased but not as efficient as the parametric methods. The half-widths of CIs mirrored local average errors. Conclusion: we recommend using the Clopper-Pearson mid-P CI for the estimation of a proportion except for observed-theoretical proportion comparison under controlled experimental conditions in which the Clopper-Pearson CI may be better.