Unknown Delay for Adversarial Bandit Setting with Multiple Play

TL;DR

This paper studies the challenge of unknown delays in adversarial multi-armed bandit problems where multiple arms are selected simultaneously, proposing a new algorithm with near-optimal regret bounds.

Contribution

It introduces the DEXP3.M algorithm tailored for multiple play scenarios with unknown delays, extending existing single-play methods to more complex applications.

Findings

The DEXP3.M algorithm achieves regret bounds close to single-play settings.

The proposed method effectively handles feedback delay and multiple arm selection.

Experimental results demonstrate improved performance over baseline algorithms.

Abstract

This paper addresses the problem of unknown delays in adversarial multi-armed bandit (MAB) with multiple play. Existing work on similar game setting focused on only the case where the learner selects an arm in each round. However, there are lots of applications in robotics where a learner needs to select more than one arm per round. It is therefore worthwhile to investigate the effect of delay when multiple arms are chosen. The multiple arms chosen per round in this setting are such that they experience the same amount of delay. There can be an aggregation of feedback losses from different combinations of arms selected at different rounds, and the learner is faced with the challenge of associating the feedback losses to the arms producing them. To address this problem, this paper proposes a delayed exponential, exploitation and exploration for multiple play (DEXP3.M) algorithm. The regret bound is only slightly worse than the regret of DEXP3 already proposed for the single play setting with unknown delay.