# Multi-player Multi-armed Bandits for Stable Allocation in Heterogeneous Ad-Hoc Networks

## TL;DR

This paper introduces a decentralized multi-armed bandit algorithm for stable, collision-free spectrum allocation in ultra-dense, heterogeneous ad-hoc networks, addressing challenges of unknown channel qualities and dynamic user entry and exit.

## Contribution

It proposes a novel low-complexity, epoch-less distributed algorithm for stable orthogonal allocation in both static and dynamic networks, with proven convergence.

## Key findings

- Algorithm achieves stable orthogonal allocation in finite time.
- Validated through extensive simulations showing effectiveness.
- Handles heterogeneity and dynamics without inter-user communication.

## Abstract

Next generation networks are expected to be ultradense and aim to explore spectrum sharing paradigm that allows users to communicate in licensed, shared as well as unlicensed spectrum. Such ultra-dense networks will incur significant signaling load at base stations leading to a negative effect on spectrum and energy efficiency. To minimize signaling overhead, an adhoc approach is being considered for users communicating in the unlicensed and shared spectrums. For such users, decisions need to be completely decentralized as: 1) No communication between users and signaling from the base station is possible which necessitates independent channel selection at each user. A collision occurs when multiple users transmit simultaneously on the same channel, 2) Channel qualities may be heterogeneous, i.e., they are not same across all users, and moreover, are unknown, and 3) The network could be dynamic where users can enter or leave anytime. We develop a multi-armed bandit based distributed algorithm for static networks and extend it for the dynamic networks. The algorithms aim to achieve stable orthogonal allocation (SOC) in finite time and meet the above three constraints with two novel characteristics: 1) Low complexity narrowband radio compared to wideband radio in existing works, and 2) Epoch-less approach for dynamic networks. We establish a convergence of our algorithms to SOC and validate via extensive simulation experiments.