Series Expansion of Probability of Correct Selection for Improved Finite Budget Allocation in Ranking and Selection

TL;DR

This paper develops a refined finite-sample approximation for the probability of correct selection in ranking and selection, introducing a new allocation policy that improves performance in limited simulation budgets.

Contribution

It introduces a Bahadur-Rao type expansion for PCS, improving finite-sample accuracy, and proposes a novel finite budget allocation policy (FCBA) for better sampling efficiency.

Findings

FCBA outperforms traditional methods in toy examples.

Refined expansion addresses non-monotonic PCS behavior in low-confidence scenarios.

Enhanced PCS approximation improves finite sample performance.

Abstract

This paper addresses the challenge of improving finite sample performance in Ranking and Selection by developing a Bahadur-Rao type expansion for the Probability of Correct Selection (PCS). While traditional large deviations approximations captures PCS behavior in the asymptotic regime, they can lack precision in finite sample settings. Our approach enhances PCS approximation under limited simulation budgets, providing more accurate characterization of optimal sampling ratios and optimality conditions dependent of budgets. Algorithmically, we propose a novel finite budget allocation (FCBA) policy, which sequentially estimates the optimality conditions and accordingly balances the sampling ratios. We illustrate numerically on toy examples that our FCBA policy achieves superior PCS performance compared to tested traditional methods. As an extension, we note that the non-monotonic PCS behavior described in the literature for low-confidence scenarios can be attributed to the negligence of simultaneous incorrect binary comparisons in PCS approximations. We provide a refined expansion and a tailored allocation strategy to handle low-confidence scenarios, addressing the non-monotonicity issue.