Multi-User Multi-Armed Bandits for Uncoordinated Spectrum Access

TL;DR

This paper develops algorithms for uncoordinated spectrum access using multi-user multi-armed bandit frameworks, addressing unknown user counts, collisions, and both stochastic and adversarial reward settings, with proven regret guarantees.

Contribution

It introduces novel algorithms for multi-user spectrum access that handle unknown user numbers, collisions, and dynamic environments, with theoretical regret bounds in stochastic and adversarial cases.

Findings

System-wide constant regret in stochastic setting

Sub-linear regret of order O(T^{3/4}) in adversarial setting

Algorithms extend to dynamic user scenarios with sub-linear regret

Abstract

A multi-user multi-armed bandit (MAB) framework is used to develop algorithms for uncoordinated spectrum access. The number of users is assumed to be unknown to each user. A stochastic setting is first considered, where the rewards on a channel are the same for each user. In contrast to prior work, it is assumed that the number of users can possibly exceed the number of channels, and that rewards can be non-zero even under collisions. The proposed algorithm consists of an estimation phase and an allocation phase. It is shown that if every user adopts the algorithm, the system wide regret is constant with time with high probability. The regret guarantees hold for any number of users and channels, in particular, even when the number of users is less than the number of channels. Next, an adversarial multi-user MAB framework is considered, where the rewards on the channels are user-dependent. It is assumed that the number of users is less than the number of channels, and that the users receive zero reward on collision. The proposed algorithm combines the Exp3.P algorithm developed in prior work for single user adversarial bandits with a collision resolution mechanism to achieve sub-linear regret. It is shown that if every user employs the proposed algorithm, the system wide regret is of the order $O(T^\frac{3}{4})$ over a horizon of time $T$. The algorithms in both stochastic and adversarial scenarios are extended to the dynamic case where the number of users in the system evolves over time and are shown to lead to sub-linear regret.