Combinatorial Bandits for Maximum Value Reward Function under Max Value-Index Feedback

TL;DR

This paper introduces a new combinatorial bandit problem with a novel feedback structure, proposing an algorithm with regret bounds comparable to more informative feedback settings, and demonstrates its effectiveness through experiments.

Contribution

It presents a new feedback structure for combinatorial bandits, along with an algorithm and regret analysis for maximum value reward functions.

Findings

The algorithm achieves $O((k/\Delta)\log(T))$ regret bound.

The regret bounds are comparable to semi-bandit feedback scenarios.

Experimental results confirm the algorithm's effectiveness.

Abstract

We consider a combinatorial multi-armed bandit problem for maximum value reward function under maximum value and index feedback. This is a new feedback structure that lies in between commonly studied semi-bandit and full-bandit feedback structures. We propose an algorithm and provide a regret bound for problem instances with stochastic arm outcomes according to arbitrary distributions with finite supports. The regret analysis rests on considering an extended set of arms, associated with values and probabilities of arm outcomes, and applying a smoothness condition. Our algorithm achieves a $O((k/\Delta)\log(T))$ distribution-dependent and a $\tilde{O}(\sqrt{T})$ distribution-independent regret where $k$ is the number of arms selected in each round, $\Delta$ is a distribution-dependent reward gap and $T$ is the horizon time. Perhaps surprisingly, the regret bound is comparable to previously-known bound under more informative semi-bandit feedback. We demonstrate the effectiveness of our algorithm through experimental results.