Weighted sampling without replacement

TL;DR

This paper extends the comparison of concentration properties between sampling with and without replacement to non-uniform weights, using couplings to derive new concentration inequalities and address questions in Polya urn sampling.

Contribution

It introduces a coupling approach for non-uniform weighted sampling, deriving Chernoff-type bounds and sub-Gaussian inequalities, and answers a question on Polya urn sampling.

Findings

Coupling induces a submartingale for cumulative values with ordered weights.

Chernoff-type upper-tail bounds transfer from sampling with to without replacement.

Established sub-Gaussian concentration inequalities for general weights.

Abstract

Comparing concentration properties of uniform sampling with and without replacement has a long history which can be traced back to the pioneer work of Hoeffding (1963). The goal of this short note is to extend this comparison to the case of non-uniform weights, using a coupling between the two samples. When the items' weights are arranged in the same order as their values, we show that the induced coupling for the cumulative values is a submartingale coupling. As a consequence, the powerful Chernoff-type upper-tail estimates known for sampling with replacement automatically transfer to the case of sampling without replacement. For general weights, we use the same coupling to establish a sub-Gaussian concentration inequality. We also construct another martingale coupling which allows us to answer a question raised by Luh and Pippenger (2014) on sampling in Polya urns with different replacement numbers.