Quantum exploration algorithms for multi-armed bandits

TL;DR

This paper introduces a quantum algorithm for multi-armed bandit problems that achieves a quadratic speedup over classical methods by efficiently identifying the best arm using quantum amplitude amplification techniques.

Contribution

It presents the first quantum algorithm with provable speedup for best-arm identification in multi-armed bandits, along with matching lower bounds.

Findings

Quantum algorithm achieves quadratic speedup in query complexity.

Matching quantum lower bounds established.

Algorithm efficiently identifies the best arm with fixed confidence.

Abstract

Identifying the best arm of a multi-armed bandit is a central problem in bandit optimization. We study a quantum computational version of this problem with coherent oracle access to states encoding the reward probabilities of each arm as quantum amplitudes. Specifically, we show that we can find the best arm with fixed confidence using $\tilde{O}\bigl(\sqrt{\sum_{i=2}^n\Delta^{\smash{-2}}_i}\bigr)$ quantum queries, where $\Delta_{i}$ represents the difference between the mean reward of the best arm and the $i^\text{th}$-best arm. This algorithm, based on variable-time amplitude amplification and estimation, gives a quadratic speedup compared to the best possible classical result. We also prove a matching quantum lower bound (up to poly-logarithmic factors).