Significance improvement by randomized test in random sampling without replacement

TL;DR

This paper introduces a randomized testing method for hypothesis testing in sampling without replacement, significantly improving the power over deterministic tests in this setting.

Contribution

It proposes a novel randomized test with a randomization parameter for upper confidence limits in sampling without replacement, enhancing test performance.

Findings

Randomized tests outperform deterministic tests in sampling without replacement.

The method provides tighter confidence bounds for the expectation of the additional variable.

Significant improvement in hypothesis testing accuracy at given significance levels.

Abstract

This paper studies one-sided hypothesis testing under random sampling without replacement. That is, when $n+1$ binary random variables $X_1,\ldots, X_{n+1}$ are subject to a permutation invariant distribution and $n$ binary random variables $X_1,\ldots, X_{n}$ are observed, we have proposed randomized tests with a randomization parameter for the upper confidence limit of the expectation of the $(n+1)$th random variable $X_{n+1}$ under a given significance level $\delta>0$. Our proposed randomized test significantly improves over deterministic test unlike random sampling with replacement.