Stochastically Constrained Best Arm Identification with Thompson Sampling

TL;DR

This paper extends Thompson sampling to the problem of best arm identification under stochastic constraints, proposing a new algorithm with proven asymptotic optimality and demonstrating superior empirical performance.

Contribution

It introduces the first Thompson sampling-based algorithm for constrained best arm identification and proves its asymptotic optimality.

Findings

Algorithm achieves asymptotic optimality in posterior convergence rate

Numerical examples show superior performance over existing methods

First extension of Thompson sampling to stochastic constrained optimization

Abstract

We consider the problem of the best arm identification in the presence of stochastic constraints, where there is a finite number of arms associated with multiple performance measures. The goal is to identify the arm that optimizes the objective measure subject to constraints on the remaining measures. We will explore the popular idea of Thompson sampling (TS) as a means to solve it. To the best of our knowledge, it is the first attempt to extend TS to this problem. We will design a TS-based sampling algorithm, establish its asymptotic optimality in the rate of posterior convergence, and demonstrate its superior performance using numerical examples.