Multi-Player Multi-Armed Bandits with Finite Shareable Resources Arms: Learning Algorithms & Applications

TL;DR

This paper introduces a novel multi-player multi-armed bandit model with shareable resources, proposing algorithms with logarithmic regret bounds and demonstrating applications in wireless networking and edge computing.

Contribution

It extends MMAB models to include shareable resources with unknown parameters and develops algorithms for different feedback types, achieving tight regret bounds.

Findings

Algorithms achieve logarithmic regret in the number of rounds.

Simulations validate the algorithms' effectiveness in practical scenarios.

Applications demonstrated in wireless networking and edge computing.

Abstract

Multi-player multi-armed bandits (MMAB) study how decentralized players cooperatively play the same multi-armed bandit so as to maximize their total cumulative rewards. Existing MMAB models mostly assume when more than one player pulls the same arm, they either have a collision and obtain zero rewards, or have no collision and gain independent rewards, both of which are usually too restrictive in practical scenarios. In this paper, we propose an MMAB with shareable resources as an extension to the collision and non-collision settings. Each shareable arm has finite shareable resources and a "per-load" reward random variable, both of which are unknown to players. The reward from a shareable arm is equal to the "per-load" reward multiplied by the minimum between the number of players pulling the arm and the arm's maximal shareable resources. We consider two types of feedback: sharing demand information (SDI) and sharing demand awareness (SDA), each of which provides different signals of resource sharing. We design the DPE-SDI and SIC-SDA algorithms to address the shareable arm problem under these two cases of feedback respectively and prove that both algorithms have logarithmic regrets that are tight in the number of rounds. We conduct simulations to validate both algorithms' performance and show their utilities in wireless networking and edge computing.