Binomial confidence intervals for rare events: importance of defining margin of error relative to magnitude of proportion

TL;DR

This paper evaluates the performance of four common confidence interval estimators for rare event proportions, emphasizing the importance of defining margin of error relative to the proportion's magnitude to ensure accurate and efficient estimation.

Contribution

It introduces a relative margin of error framework for assessing confidence intervals of rare event proportions, guiding practical selection of estimators and margin values.

Findings

All four estimators perform similarly under the new framework.

Relative margin of error values can optimize coverage and sample size.

The scheme is validated analytically, via simulation, and through literature application.

Abstract

Confidence interval performance is typically assessed in terms of two criteria: coverage probability and interval width (or margin of error). In this paper, we assess the performance of four common proportion interval estimators: the Wald, Clopper-Pearson (exact), Wilson and Agresti-Coull, in the context of rare-event probabilities. We define the interval precision in terms of a relative margin of error which ensures consistency with the magnitude of the proportion. Thus, confidence interval estimators are assessed in terms of achieving a desired coverage probability whilst simultaneously satisfying the specified relative margin of error. We illustrate the importance of considering both coverage probability and relative margin of error when estimating rare-event proportions, and show that within this framework, all four interval estimators perform somewhat similarly for a given sample size and confidence level. We identify relative margin of error values that result in satisfactory coverage whilst being conservative in terms of sample size requirements, and hence suggest a range of values that can be adopted in practice. The proposed relative margin of error scheme is evaluated analytically, by simulation, and by application to a number of recent studies from the literature.