Combinatorial semi-bandit with known covariance

TL;DR

This paper introduces a new algorithm for the combinatorial semi-bandit problem that accounts for dependencies among arms, achieving near-optimal performance by adapting to the covariance structure.

Contribution

It develops a method to quantify arm dependencies and designs an adaptive algorithm based on linear regression, improving over prior independence assumptions.

Findings

The algorithm is proven to be nearly optimal compared to a new lower bound.

It effectively adapts to the dependency structure of arms.

Performance matches theoretical lower bounds up to poly-logarithmic factors.

Abstract

The combinatorial stochastic semi-bandit problem is an extension of the classical multi-armed bandit problem in which an algorithm pulls more than one arm at each stage and the rewards of all pulled arms are revealed. One difference with the single arm variant is that the dependency structure of the arms is crucial. Previous works on this setting either used a worst-case approach or imposed independence of the arms. We introduce a way to quantify the dependency structure of the problem and design an algorithm that adapts to it. The algorithm is based on linear regression and the analysis develops techniques from the linear bandit literature. By comparing its performance to a new lower bound, we prove that it is optimal, up to a poly-logarithmic factor in the number of pulled arms.