Cooperative Multi-Agent Reinforcement Learning for Low-Level Wireless Communication

TL;DR

This paper introduces a reinforcement learning approach for two agents to autonomously develop low-level wireless communication protocols, potentially increasing flexibility and efficiency in spectrum use.

Contribution

It presents a novel decentralized reinforcement learning method using policy gradients to discover wireless communication schemes from scratch.

Findings

Agents learn sophisticated communication protocols

The approach demonstrates effective spectrum utilization

Reinforcement learning enables flexible wireless communication

Abstract

Traditional radio systems are strictly co-designed on the lower levels of the OSI stack for compatibility and efficiency. Although this has enabled the success of radio communications, it has also introduced lengthy standardization processes and imposed static allocation of the radio spectrum. Various initiatives have been undertaken by the research community to tackle the problem of artificial spectrum scarcity by both making frequency allocation more dynamic and building flexible radios to replace the static ones. There is reason to believe that just as computer vision and control have been overhauled by the introduction of machine learning, wireless communication can also be improved by utilizing similar techniques to increase the flexibility of wireless networks. In this work, we pose the problem of discovering low-level wireless communication schemes ex-nihilo between two agents in a fully decentralized fashion as a reinforcement learning problem. Our proposed approach uses policy gradients to learn an optimal bi-directional communication scheme and shows surprisingly sophisticated and intelligent learning behavior. We present the results of extensive experiments and an analysis of the fidelity of our approach.