The Emergence of Wireless MAC Protocols with Multi-Agent Reinforcement Learning

TL;DR

This paper introduces a multi-agent reinforcement learning framework using MADDPG for developing MAC protocols in wireless networks, enabling cooperation between base stations and user equipment without prior protocol agreements.

Contribution

It presents a novel RL-based framework for wireless MAC protocol design that learns cooperation and signaling policies without predefined rules.

Findings

Outperforms contention-free baseline in goodput

Effective learning of channel access and signaling policies

Cooperation between agents improves network performance

Abstract

In this paper, we propose a new framework, exploiting the multi-agent deep deterministic policy gradient (MADDPG) algorithm, to enable a base station (BS) and user equipment (UE) to come up with a medium access control (MAC) protocol in a multiple access scenario. In this framework, the BS and UEs are reinforcement learning (RL) agents that need to learn to cooperate in order to deliver data. The network nodes can exchange control messages to collaborate and deliver data across the network, but without any prior agreement on the meaning of the control messages. In such a framework, the agents have to learn not only the channel access policy, but also the signaling policy. The collaboration between agents is shown to be important, by comparing the proposed algorithm to ablated versions where either the communication between agents or the central critic is removed. The comparison with a contention-free baseline shows that our framework achieves a superior performance in terms of goodput and can effectively be used to learn a new protocol.