Threshold rules for online sample selection

TL;DR

This paper studies online sample selection where samples with qualities arrive sequentially, and simple threshold rules can optimally balance selection rate and aggregate quality across various distributions.

Contribution

It demonstrates that simple, oblivious threshold rules can achieve optimal tradeoffs in online sample selection problems, often universally across different distributions.

Findings

Threshold rules achieve optimal tradeoffs in selection rate and quality.

Some threshold rules are optimal across broad classes of distributions.

Simple rules can be as effective as complex strategies in online selection.

Abstract

We consider the following sample selection problem. We observe in an online fashion a sequence of samples, each endowed by a quality. Our goal is to either select or reject each sample, so as to maximize the aggregate quality of the subsample selected so far. There is a natural trade-off here between the rate of selection and the aggregate quality of the subsample. We show that for a number of such problems extremely simple and oblivious "threshold rules" for selection achieve optimal tradeoffs between rate of selection and aggregate quality in a probabilistic sense. In some cases we show that the same threshold rule is optimal for a large class of quality distributions and is thus oblivious in a strong sense.