Multiple-play Stochastic Bandits with Prioritized Arm Capacity Sharing

TL;DR

This paper introduces a novel multi-play stochastic bandit model with prioritized resource sharing, providing theoretical regret bounds and an efficient algorithm for optimal play allocation in complex resource management scenarios.

Contribution

It develops a new bandit model with prioritized capacity sharing, derives regret bounds, and proposes algorithms for optimal resource allocation under this model.

Findings

Proved regret lower bounds for the model.

Designed an algorithm with regret bounds matching lower bounds.

Addressed complex nonlinear utility optimization challenges.

Abstract

This paper proposes a variant of multiple-play stochastic bandits tailored to resource allocation problems arising from LLM applications, edge intelligence, etc. The model is composed of $M$ arms and $K$ plays. Each arm has a stochastic number of capacities, and each unit of capacity is associated with a reward function. Each play is associated with a priority weight. When multiple plays compete for the arm capacity, the arm capacity is allocated in a larger priority weight first manner. Instance independent and instance dependent regret lower bounds of $\Omega( \alpha_1 \sigma \sqrt{KM T} )$ and $\Omega(\alpha_1 \sigma^2 \frac{M}{\Delta} \ln T)$ are proved, where $\alpha_1$ is the largest priority weight and $\sigma$ characterizes the reward tail. When model parameters are given, we design an algorithm named \texttt{MSB-PRS-OffOpt} to locate the optimal play allocation policy with a computational complexity of $O(MK^3)$. Utilizing \texttt{MSB-PRS-OffOpt} as a subroutine, an approximate upper confidence bound (UCB) based algorithm is designed, which has instance independent and instance dependent regret upper bounds matching the corresponding lower bound up to factors of $ \sqrt{K \ln KT }$ and $\alpha_1 K^2$ respectively. To this end, we address nontrivial technical challenges arising from optimizing and learning under a special nonlinear combinatorial utility function induced by the prioritized resource sharing mechanism.