Cost-Efficient Fixed-Width Confidence Intervals for the Difference of Two Bernoulli Proportions

TL;DR

This paper introduces three cost-efficient fixed-width confidence interval methods for the difference of two Bernoulli proportions, optimizing sampling costs while maintaining coverage and width constraints through simulation-based evaluation.

Contribution

It proposes three novel fixed-width CI construction methods tailored for differing observation costs, demonstrating significant cost savings and efficiency improvements over baseline approaches.

Findings

Up to 50% cost savings with the two-stage procedure

Sequential methods achieve nominal coverage and desired width

Sequential-batching is computationally more efficient

Abstract

We study properties of confidence intervals (CIs) for the difference of two Bernoulli distributions' success parameters, $p_x - p_y$, in the case where the goal is to obtain a CI of a given half-width while minimizing sampling costs when the observation costs may be different between the two distributions. Assuming that we are provided with preliminary estimates of the success parameters, we propose three different methods for constructing fixed-width CIs: (i) a two-stage sampling procedure, (ii) a sequential method that carries out sampling in batches, and (iii) an $\ell$-stage "look-ahead" procedure. We use Monte Carlo simulation to show that, under diverse success probability and observation cost scenarios, our proposed algorithms obtain significant cost savings versus their baseline counterparts (up to 50\% for the two-stage procedure, up to 15\% for the sequential methods). Furthermore, for the battery of scenarios under study, our sequential-batches and $\ell$-stage "look-ahead" procedures approximately obtain the nominal coverage while also meeting the desired width requirement. Our sequential-batching method turned out to be more efficient than the "look-ahead" method from a computational standpoint, with average running times at least an order-of-magnitude faster over all the scenarios tested.