An Algorithm for Fixed Budget Best Arm Identification with Combinatorial Exploration

TL;DR

This paper introduces a novel algorithm for fixed budget best arm identification in multi-armed bandits, allowing subset plays and using group testing to efficiently identify the best arm with theoretical guarantees.

Contribution

It proposes a new algorithm that constructs groups and applies likelihood ratio tests, providing error bounds and outperforming existing methods in certain scenarios.

Findings

Derives an upper bound for error probability based on a new hardness parameter H_4.

Demonstrates cases where the algorithm outperforms state-of-the-art single-play algorithms.

Introduces a group testing approach for combinatorial exploration in bandit problems.

Abstract

We consider the best arm identification (BAI) problem in the $K-$armed bandit framework with a modification - the agent is allowed to play a subset of arms at each time slot instead of one arm. Consequently, the agent observes the sample average of the rewards of the arms that constitute the probed subset. Several trade-offs arise here - e.g., sampling a larger number of arms together results in a wider view of the environment, while sampling fewer arms enhances the information about individual reward distributions. Furthermore, grouping a large number of suboptimal arms together albeit reduces the variance of the reward of the group, it may enhance the group mean to make it close to that containing the optimal arm. To solve this problem, we propose an algorithm that constructs $\log_2 K$ groups and performs a likelihood ratio test to detect the presence of the best arm in each of these groups. Then a Hamming decoding procedure determines the unique best arm. We derive an upper bound for the error probability of the proposed algorithm based on a new hardness parameter $H_4$. Finally, we demonstrate cases under which it outperforms the state-of-the-art algorithms for the single play case.