Learning-Based Channel Access in Wi-Fi: A Multi-Armed Bandit Approach

TL;DR

This paper explores reinforcement learning, specifically multi-armed bandit algorithms, to improve Wi-Fi channel access by making it more adaptive to dynamic network conditions, leading to better spectrum utilization.

Contribution

It introduces a multi-armed bandit framework for Wi-Fi medium access control, comparing single-agent and multi-agent approaches, and highlights the benefits of contextual information and decentralized learning.

Findings

Multi-agent architectures converge faster than single-agent ones.

Contextual bandit algorithms outperform non-contextual methods.

Decentralized learners can implicitly coordinate but may cause policy-chasing dynamics.

Abstract

Due to its static protocol design, IEEE 802.11 (aka Wi-Fi) channel access lacks adaptability to address dynamic network conditions, resulting in inefficient spectrum utilization, unnecessary contention, and packet collisions. This paper investigates reinforcement learning (RL) solutions to optimize Wi-Fi's medium access control (MAC). In particular, a multi-armed bandit (MAB) framework is proposed for dynamic channel access (including both the primary channel and channel width) and contention window (CW) adjustment. In this setting, we study relevant learning design principles such as adopting joint or factorial action spaces (handled by a single agent (SA) and multiple agents (MA), respectively) and the importance of incorporating contextual information. Our simulation results show that cooperative MA architectures converge faster than their SA counterparts, as agents operate over smaller action spaces. Another key insight is that contextual MAB algorithms consistently outperform non-contextual ones, highlighting the value of leveraging side information in action selection. Moreover, in multi-player settings, results demonstrate that decentralized learners can achieve implicit coordination, although their greediness may degrade coexisting networks' performance and induce policy-chasing dynamics. Overall, these findings demonstrate that (contextual) MAB-based learning offers a practical and adaptive alternative to static IEEE 802.11 protocols, enabling more efficient and intelligent spectrum utilization.