Adversarial Combinatorial Semi-bandits with Graph Feedback

TL;DR

This paper extends combinatorial semi-bandits to include graph feedback, deriving optimal regret bounds that interpolate between full information and semi-bandit feedback, with new technical insights on action realization and regret analysis.

Contribution

It introduces a framework for graph feedback in combinatorial semi-bandits, establishes tight regret bounds, and proposes novel action realization techniques.

Findings

Optimal regret scales as tildeTheta(Ssqrt{T}+sqrt{alpha ST})

Convexified action realization with negative correlations improves performance

OSMD with convexified actions in expectation is suboptimal

Abstract

In combinatorial semi-bandits, a learner repeatedly selects from a combinatorial decision set of arms, receives the realized sum of rewards, and observes the rewards of the individual selected arms as feedback. In this paper, we extend this framework to include \emph{graph feedback}, where the learner observes the rewards of all neighboring arms of the selected arms in a feedback graph $G$. We establish that the optimal regret over a time horizon $T$ scales as $\widetilde{\Theta}(S\sqrt{T}+\sqrt{\alpha ST})$, where $S$ is the size of the combinatorial decisions and $\alpha$ is the independence number of $G$. This result interpolates between the known regrets $\widetilde\Theta(S\sqrt{T})$ under full information (i.e., $G$ is complete) and $\widetilde\Theta(\sqrt{KST})$ under the semi-bandit feedback (i.e., $G$ has only self-loops), where $K$ is the total number of arms. A key technical ingredient is to realize a convexified action using a random decision vector with negative correlations. We also show that online stochastic mirror descent (OSMD) that only realizes convexified actions in expectation is suboptimal. In addition, we describe the problem of \emph{combinatorial semi-bandits with general capacity} and apply our results to derive an improved regret upper bound, which may be of independent interest.