Stateless Reinforcement Learning for Multi-Agent Systems: the Case of Spectrum Allocation in Dynamic Channel Bonding WLANs

TL;DR

This paper advocates for using lightweight, stateless reinforcement learning methods like multi-armed bandits for rapid, decentralized spectrum allocation in dynamic WLANs, challenging the need for complex RL models.

Contribution

It justifies the use of simple, stateless RL approaches over complex models for efficient, real-time spectrum management in decentralized wireless networks.

Findings

Stateless RL methods enable faster adaptation in dynamic environments.

Complex RL algorithms may not be necessary for effective spectrum allocation.

Lightweight multi-armed bandits outperform complex models in realistic scenarios.

Abstract

Spectrum allocation in the form of primary channel and bandwidth selection is a key factor for dynamic channel bonding (DCB) wireless local area networks (WLANs). To cope with varying environments, where networks change their configurations on their own, the wireless community is looking towards solutions aided by machine learning (ML), and especially reinforcement learning (RL) given its trial-and-error approach. However, strong assumptions are normally made to let complex RL models converge to near-optimal solutions. Our goal with this paper is two-fold: justify in a comprehensible way why RL should be the approach for wireless networks problems like decentralized spectrum allocation, and call into question whether the use of complex RL algorithms helps the quest of rapid learning in realistic scenarios. We derive that stateless RL in the form of lightweight multi-armed-bandits (MABs) is an efficient solution for rapid adaptation avoiding the definition of extensive or meaningless RL states.