Towards Multi-agent Reinforcement Learning for Wireless Network Protocol Synthesis

TL;DR

This paper introduces a multi-agent reinforcement learning framework for wireless network medium access control, enabling adaptive, high-throughput, and load-agnostic protocols that outperform classical methods.

Contribution

It develops a scalable RL-based MAC protocol that adapts to various network topologies and dynamic loads, improving throughput and flexibility over traditional approaches.

Findings

Achieves maximum throughput at optimal load conditions.

Retains high throughput under increased network load.

Adapts to time-varying traffic conditions online.

Abstract

This paper proposes a multi-agent reinforcement learning based medium access framework for wireless networks. The access problem is formulated as a Markov Decision Process (MDP), and solved using reinforcement learning with every network node acting as a distributed learning agent. The solution components are developed step by step, starting from a single-node access scenario in which a node agent incrementally learns to control MAC layer packet loads for reining in self-collisions. The strategy is then scaled up for multi-node fully-connected scenarios by using more elaborate reward structures. It also demonstrates preliminary feasibility for more general partially connected topologies. It is shown that by learning to adjust MAC layer transmission probabilities, the protocol is not only able to attain theoretical maximum throughput at an optimal load, but unlike classical approaches, it can also retain that maximum throughput at higher loading conditions. Additionally, the mechanism is agnostic to heterogeneous loading while preserving that feature. It is also shown that access priorities of the protocol across nodes can be parametrically adjusted. Finally, it is also shown that the online learning feature of reinforcement learning is able to make the protocol adapt to time-varying loading conditions.