Near-Optimal Regret Bounds for Contextual Combinatorial Semi-Bandits with Linear Payoff Functions

TL;DR

This paper improves regret bounds for the contextual combinatorial semi-bandit problem with linear payoffs, achieving near-optimal bounds that depend on feature dimension, number of arms, and rounds, applicable to practical scenarios.

Contribution

The authors refine upper and lower regret bounds for the problem, introducing an algorithm with optimal regret bounds under general constraints and demonstrating broader applicability.

Findings

C${}^2$UCB algorithm has optimal regret bound $ ilde{O}(d ext{sqrt}(kT) + dk)$ for partition matroids.

Modified reward estimates lead to optimal regret bounds for general constraints.

Numerical experiments confirm theoretical regret bounds and practical effectiveness.

Abstract

The contextual combinatorial semi-bandit problem with linear payoff functions is a decision-making problem in which a learner chooses a set of arms with the feature vectors in each round under given constraints so as to maximize the sum of rewards of arms. Several existing algorithms have regret bounds that are optimal with respect to the number of rounds $T$. However, there is a gap of $\tilde{O}(\max(\sqrt{d}, \sqrt{k}))$ between the current best upper and lower bounds, where $d$ is the dimension of the feature vectors, $k$ is the number of the chosen arms in a round, and $\tilde{O}(\cdot)$ ignores the logarithmic factors. The dependence of $k$ and $d$ is of practical importance because $k$ may be larger than $T$ in real-world applications such as recommender systems. In this paper, we fill the gap by improving the upper and lower bounds. More precisely, we show that the C${}^2$UCB algorithm proposed by Qin, Chen, and Zhu (2014) has the optimal regret bound $\tilde{O}(d\sqrt{kT} + dk)$ for the partition matroid constraints. For general constraints, we propose an algorithm that modifies the reward estimates of arms in the C${}^2$UCB algorithm and demonstrate that it enjoys the optimal regret bound for a more general problem that can take into account other objectives simultaneously. We also show that our technique would be applicable to related problems. Numerical experiments support our theoretical results and considerations.