Multi-armed quantum bandits: Exploration versus exploitation when learning properties of quantum states

TL;DR

This paper explores the balance between exploration and exploitation in learning properties of unknown quantum states, providing theoretical bounds on regret and strategies for optimal learning in quantum bandit settings.

Contribution

It introduces the first information-theoretic lower bounds on regret in quantum bandits and proposes optimal strategies for finite actions and mixed states.

Findings

Regret scales at least as the square root of the number of rounds

Lower bounds depend on number of actions and space dimension

Strategies are optimal for finite arms and mixed states

Abstract

We initiate the study of tradeoffs between exploration and exploitation in online learning of properties of quantum states. Given sequential oracle access to an unknown quantum state, in each round, we are tasked to choose an observable from a set of actions aiming to maximize its expectation value on the state (the reward). Information gained about the unknown state from previous rounds can be used to gradually improve the choice of action, thus reducing the gap between the reward and the maximal reward attainable with the given action set (the regret). We provide various information-theoretic lower bounds on the cumulative regret that an optimal learner must incur, and show that it scales at least as the square root of the number of rounds played. We also investigate the dependence of the cumulative regret on the number of available actions and the dimension of the underlying space. Moreover, we exhibit strategies that are optimal for bandits with a finite number of arms and general mixed states.