Review on Ranking and Selection: A New Perspective

TL;DR

This paper reviews the development of ranking-and-selection methods over 70 years, focusing on their theoretical foundations, practical applications, and categorization into fixed-precision and fixed-budget procedures.

Contribution

It introduces a new categorization of R&S procedures based on their methodological foundations, contrasting hypothesis testing and dynamic programming approaches.

Findings

Different R&S procedures are built on hypothesis testing or dynamic programming.

The paper maps well-known procedures into these two categories.

It discusses practical applications and future research directions.

Abstract

In this paper, we briefly review the development of ranking-and-selection (R&S) in the past 70 years, especially the theoretical achievements and practical applications in the last 20 years. Different from the frequentist and Bayesian classifications adopted by Kim and Nelson (2006b) and Chick (2006) in their review articles, we categorize existing R&S procedures into fixed-precision and fixed-budget procedures, as in Hunter and Nelson (2017). We show that these two categories of procedures essentially differ in the underlying methodological formulations, i.e., they are built on hypothesis testing and dynamic-programming, respectively. In light of this variation, we review in detail some well-known procedures in the literature and show how they fit into these two formulations. In addition, we discuss the use of R&S procedures in solving various practical problems and propose what we think are the important research questions in the field.