Multi-user Communication Networks: A Coordinated Multi-armed Bandit Approach

TL;DR

This paper introduces a novel approach combining multi-armed bandit learning with lightweight coordination to optimize resource sharing in multi-user communication networks, ensuring stability and maximal utilization.

Contribution

It presents a new system-wide algorithm that guarantees convergence to stable resource allocations with minimal signaling overhead, applicable to both fixed and dynamic user scenarios.

Findings

Proves convergence to stable configurations in both scenarios

Achieves higher resource utilization than existing methods

Demonstrates robustness across diverse experimental setups

Abstract

Communication networks shared by many users are a widespread challenge nowadays. In this paper we address several aspects of this challenge simultaneously: learning unknown stochastic network characteristics, sharing resources with other users while keeping coordination overhead to a minimum. The proposed solution combines Multi-Armed Bandit learning with a lightweight signalling-based coordination scheme, and ensures convergence to a stable allocation of resources. Our work considers single-user level algorithms for two scenarios: an unknown fixed number of users, and a dynamic number of users. Analytic performance guarantees, proving convergence to stable marriage configurations, are presented for both setups. The algorithms are designed based on a system-wide perspective, rather than focusing on single user welfare. Thus, maximal resource utilization is ensured. An extensive experimental analysis covers convergence to a stable configuration as well as reward maximization. Experiments are carried out over a wide range of setups, demonstrating the advantages of our approach over existing state-of-the-art methods.