Tighter Confidence Intervals under Without Replacement Sampling via Empirical Rate Functions

TL;DR

This paper develops tighter confidence intervals for the population mean under without replacement sampling by leveraging large deviation rate functions, providing both theoretical bounds and practical constructions for finite and continuous sample spaces.

Contribution

It introduces a new CI based on empirical inverse rate functions that matches theoretical lower bounds in certain regimes and extends the approach to continuous and Banach space settings.

Findings

Derived a fundamental lower bound on CI width using large deviation rate functions.

Proposed a new CI that matches the lower bound asymptotically.

Extended the methodology to continuous and Banach space sample spaces.

Abstract

We consider the problem of constructing confidence intervals (CIs) for the population mean of $N$ values $\{x_1, \ldots, x_N\} \subset \Sigma^N$ based on a random sample of size $n$, denoted by $X^n \equiv (X_1, \ldots, X_n)$, drawn uniformly without replacement (WoR). We begin by focusing on the finite alphabet ($|\Sigma| = k <\infty$) and moderate accuracy ($\log(1/\alpha_N) \gg (k+1)\log N$) regime, and derive a fundamental lower bound on the width of any level-$(1-\alpha_N)$ CI in terms of the inverse of the WoR rate functions from the theory of large deviations. Guided by this lower bound, we propose a new level-$(1-\alpha_N)$ CI using an empirical inverse rate function, and show that in certain asymptotic regimes the width of this CI matches the lower bound up to constants. We also derive a dual formulation of the inverse rate function that enables efficient computation of our proposed CI. We then move beyond the finite alphabet case and use a Bernoulli coupling idea to construct an almost sure CI for $\Sigma = [0,1]$, and a conceptually simple nonasymptotic CI for the case of $\Sigma$ being a $(2,D)$ smooth Banach space. For both finite and general alphabets, our results employ classical large deviation techniques in novel ways, thus establishing new connections between estimation under WoR sampling and the theory of large deviations.