When Combinatorial Thompson Sampling meets Approximation Regret

TL;DR

This paper advances the understanding of Combinatorial Thompson Sampling (CTS) in multi-armed bandit problems by establishing a new logarithmic regret bound under a specific oracle condition, broadening its applicability.

Contribution

It introduces the REDUCE2EXACT condition, enabling a new $ ext{O}(rac{ ext{log}(T)}{ riangle})$ regret bound for CTS beyond the greedy oracle case.

Findings

First $ ext{O}(rac{ ext{log}(T)}{ riangle})$ regret bound for CTS

The REDUCE2EXACT condition applies to many concrete examples

Extension of results to probabilistically triggered arms setting

Abstract

We study the Combinatorial Thompson Sampling policy (CTS) for combinatorial multi-armed bandit problems (CMAB), within an approximation regret setting. Although CTS has attracted a lot of interest, it has a drawback that other usual CMAB policies do not have when considering non-exact oracles: for some oracles, CTS has a poor approximation regret (scaling linearly with the time horizon $T$) [Wang and Chen, 2018]. A study is then necessary to discriminate the oracles on which CTS could learn. This study was started by Kong et al. [2021]: they gave the first approximation regret analysis of CTS for the greedy oracle, obtaining an upper bound of order $\mathcal{O}(\log(T)/\Delta^2)$, where $\Delta$ is some minimal reward gap. In this paper, our objective is to push this study further than the simple case of the greedy oracle. We provide the first $\mathcal{O}(\log(T)/\Delta)$ approximation regret upper bound for CTS, obtained under a specific condition on the approximation oracle, allowing a reduction to the exact oracle analysis. We thus term this condition REDUCE2EXACT, and observe that it is satisfied in many concrete examples. Moreover, it can be extended to the probabilistically triggered arms setting, thus capturing even more problems, such as online influence maximization.