Ranking and Selection as Stochastic Control

TL;DR

This paper formulates the sequential ranking and selection process as a stochastic control problem under a Bayesian framework, deriving an approximately optimal, computationally efficient allocation policy with strong optimality properties.

Contribution

It introduces a novel stochastic control formulation for Bayesian ranking and selection, deriving an approximately optimal policy with proven optimality features.

Findings

Policy is computationally efficient

Policy exhibits one-step-ahead optimality

Policy shows asymptotic optimality

Abstract

Under a Bayesian framework, we formulate the fully sequential sampling and selection decision in statistical ranking and selection as a stochastic control problem, and derive the associated Bellman equation. Using value function approximation, we derive an approximately optimal allocation policy. We show that this policy is not only computationally efficient but also possesses both one-step-ahead and asymptotic optimality for independent normal sampling distributions. Moreover, the proposed allocation policy is easily generalizable in the approximate dynamic programming paradigm.