Tight Bounds for Bandit Combinatorial Optimization

TL;DR

This paper establishes tight bounds on the regret rates in bandit combinatorial optimization, showing the growth rate as (k^{3/2}\u221a(dT)), which refutes previous conjectures and applies to key problems like the bandit shortest path.

Contribution

It proves the exact regret growth rate in bandit combinatorial optimization, resolving open problems and disproving prior conjectures about the optimal regret bounds.

Findings

Regret grows as (k^{3/2}(dT))

Disproves the conjecture that the rate is (k\u221a(dT))

Provides tight bounds for the bandit shortest path problem

Abstract

We revisit the study of optimal regret rates in bandit combinatorial optimization---a fundamental framework for sequential decision making under uncertainty that abstracts numerous combinatorial prediction problems. We prove that the attainable regret in this setting grows as $\widetilde{\Theta}(k^{3/2}\sqrt{dT})$ where $d$ is the dimension of the problem and $k$ is a bound over the maximal instantaneous loss, disproving a conjecture of Audibert, Bubeck, and Lugosi (2013) who argued that the optimal rate should be of the form $\widetilde{\Theta}(k\sqrt{dT})$. Our bounds apply to several important instances of the framework, and in particular, imply a tight bound for the well-studied bandit shortest path problem. By that, we also resolve an open problem posed by Cesa-Bianchi and Lugosi (2012).