Hierarchical Multi-Agent Reinforcement Learning-based Coordinated Spatial Reuse for Next Generation WLANs

TL;DR

This paper introduces a hierarchical multi-agent reinforcement learning approach to improve coordinated spatial reuse in dense Wi-Fi networks, significantly enhancing throughput and fairness while maintaining robustness with legacy devices.

Contribution

It presents a novel HMARL algorithm that decomposes the CSR process into polling and decision phases, optimizing station selection and power control in Wi-Fi networks.

Findings

Outperforms baseline methods in throughput and latency

Maintains robust performance with legacy APs

Enhances fairness in high-interference regions

Abstract

High-density Wi-Fi deployments often result in significant co-channel interference, which degrades overall network performance. To address this issue, coordination of multi access points (APs) has been considered to enable coordinated spatial reuse (CSR) in next generation wireless local area networks. This paper tackles the challenge of downlink spatial reuse in Wi-Fi networks, specifically in scenarios involving overlapping basic service sets, by employing hierarchical multi-agent reinforcement learning (HMARL). We decompose the CSR process into two phases, i.e., a polling phase and a decision phase, and introduce the HMARL algorithm to enable efficient CSR. To enhance training efficiency, the proposed HMARL algorithm employs a hierarchical structure, where station selection and power control are determined by a high- and low-level policy network, respectively. Simulation results demonstrate that this approach consistently outperforms baseline methods in terms of throughput and latency across various network topologies. Moreover, the algorithm exhibits robust performance when coexisting with legacy APs. Additional experiments in a representative topology further reveal that the carefully designed reward function not only maximizes the overall network throughput, but also improves fairness in transmission opportunities for APs in high-interference regions.