Sampling with Costs

TL;DR

This paper investigates optimal sampling strategies balancing quality gains against costs and measurement errors, using order statistics to inform decision-making in scenarios like mate selection and hiring.

Contribution

It introduces a framework combining costs and measurement errors into sampling strategies, providing new insights and guidelines for optimal sample size selection.

Findings

Optimal sample size depends on cost and measurement error levels.

Strategies can significantly improve decision quality compared to naive sampling.

Unexpected insights into the trade-offs between sampling effort and accuracy.

Abstract

We consider the problem of choosing the best of $n$ samples, out of a large random pool, when the sampling of each member is associated with a certain cost. The quality (worth) of the best sample clearly increases with $n$, but so do the sampling costs, and one important question is how many to sample for optimal gain (worth minus costs). If, in addition, the assessment of worth for each sample is associated with some "measurement error," the perceived best out of $n$ might not be the actual best, complicating the issue. Situations like this are typical in mate selection, job hiring, and food foraging, to name just a few. We tackle the problem by standard order statistics, yielding suggestions for optimal strategies, as well as some unexpected insights.