Best Arm Identification in Spectral Bandits

TL;DR

This paper introduces an efficient algorithm for best-arm identification in spectral bandits with graph smoothness constraints, providing an asymptotically optimal strategy and demonstrating its effectiveness through numerical experiments.

Contribution

It develops a gradient ascent algorithm to compute sample complexity and proposes an optimal strategy for BAI under graph smoothness constraints, extending prior work beyond regret minimization.

Findings

The proposed algorithm effectively computes sample complexity.

The strategy is asymptotically optimal.

Numerical experiments show efficiency and impact of smoothness constraints.

Abstract

We study best-arm identification with fixed confidence in bandit models with graph smoothness constraint. We provide and analyze an efficient gradient ascent algorithm to compute the sample complexity of this problem as a solution of a non-smooth max-min problem (providing in passing a simplified analysis for the unconstrained case). Building on this algorithm, we propose an asymptotically optimal strategy. We furthermore illustrate by numerical experiments both the strategy's efficiency and the impact of the smoothness constraint on the sample complexity. Best Arm Identification (BAI) is an important challenge in many applications ranging from parameter tuning to clinical trials. It is now very well understood in vanilla bandit models, but real-world problems typically involve some dependency between arms that requires more involved models. Assuming a graph structure on the arms is an elegant practical way to encompass this phenomenon, but this had been done so far only for regret minimization. Addressing BAI with graph constraints involves delicate optimization problems for which the present paper offers a solution.