Multi-Armed Bandits for Decentralized AP selection in Enterprise WLANs

TL;DR

This paper introduces a decentralized reinforcement learning method using Multi-Armed Bandits for AP selection in dense WiFi networks, improving load balancing and resource utilization.

Contribution

It proposes a novel Opportunistic epsilon-greedy with Stickiness approach for decentralized AP selection, enhancing convergence speed and network efficiency.

Findings

Faster convergence to optimal APs with the proposed method.

Improved network resource utilization and load balancing.

Effective in non-stationary environments with station arrivals.

Abstract

WiFi densification leads to the existence of multiple overlapping coverage areas, which allows user stations (STAs) to choose between different Access Points (APs). The standard WiFi association method makes the STAs select the AP with the strongest signal, which in many cases leads to underutilization of some APs while overcrowding others. To mitigate this situation, Reinforcement Learning techniques such as Multi-Armed Bandits can be used to dynamically learn the optimal mapping between APs and STAs, and so redistribute the STAs among the available APs accordingly. This is an especially challenging problem since the network response observed by a given STA depends on the behavior of the others, and so it is very difficult to predict without a global view of the network. In this paper, we focus on solving this problem in a decentralized way, where STAs independently explore the different APs inside their coverage range, and select the one that better satisfy their needs. To do it, we propose a novel approach called Opportunistic epsilon-greedy with Stickiness that halts the exploration when a suitable AP is found, only resuming the exploration after several unsatisfactory association rounds. With this approach, we reduce significantly the network response dynamics, improving the ability of the STAs to find a solution faster, as well as achieving a more efficient use of the network resources. We investigate how the characteristics of the scenario (position of the APs and STAs, traffic loads, and channel allocation strategies) impact the learning process and the achievable performance. We also show that not all the STAs have to implement the proposed solution to improve their performance. Finally, we study the case where stations arrive progressively to the system, showing that the considered approach is also suitable in such a non-stationary set-up.