Matroid Semi-Bandits in Sublinear Time

TL;DR

This paper introduces FasterCUCB, a new algorithm for matroid semi-bandits that achieves sublinear per-round time complexity for common matroid classes, maintaining near-optimal regret bounds.

Contribution

It proposes FasterCUCB, a computationally efficient algorithm with sublinear time per round for matroid semi-bandits, using approximate maximum-weight basis maintenance.

Findings

Achieves $O(D ext{ polylog}(K) ext{ polylog}(T))$ time for uniform, partition, and graphical matroids.

Achieves $O(D ext{sqrt{K}} ext{ polylog}(T))$ time for transversal matroids.

Maintains regret bounds comparable to CUCB, matching the asymptotic lower bound.

Abstract

We study the matroid semi-bandits problem, where at each round the learner plays a subset of $K$ arms from a feasible set, and the goal is to maximize the expected cumulative linear rewards. Existing algorithms have per-round time complexity at least $\Omega(K)$, which becomes expensive when $K$ is large. To address this computational issue, we propose FasterCUCB whose sampling rule takes time sublinear in $K$ for common classes of matroids: $O(D\text{ polylog}(K)\text{ polylog}(T))$ for uniform matroids, partition matroids, and graphical matroids, and $O(D\sqrt{K}\text{ polylog}(T))$ for transversal matroids. Here, $D$ is the maximum number of elements in any feasible subset of arms, and $T$ is the horizon. Our technique is based on dynamic maintenance of an approximate maximum-weight basis over inner-product weights. Although the introduction of an approximate maximum-weight basis presents a challenge in regret analysis, we can still guarantee an upper bound on regret as tight as CUCB in the sense that it matches the gap-dependent lower bound by Kveton et al. (2014a) asymptotically.