A Multi-Agent Reinforcement Learning Testbed for Cognitive Radio Applications

TL;DR

This paper introduces an enhanced multi-agent reinforcement learning testbed for cognitive radio applications, expanding the previous single-agent environment to support multi-agent scenarios for more realistic RF spectrum simulations.

Contribution

The paper presents the integration of multi-agent reinforcement learning into the RFRL Gym, enabling simulation of cooperative and competitive RF spectrum scenarios.

Findings

Successful implementation of multi-agent RL in RF spectrum simulation

Demonstrated the environment's capability with various RF scenarios

Enhanced the tool's robustness for future research and development

Abstract

Technological trends show that Radio Frequency Reinforcement Learning (RFRL) will play a prominent role in the wireless communication systems of the future. Applications of RFRL range from military communications jamming to enhancing WiFi networks. Before deploying algorithms for these purposes, they must be trained in a simulation environment to ensure adequate performance. For this reason, we previously created the RFRL Gym: a standardized, accessible tool for the development and testing of reinforcement learning (RL) algorithms in the wireless communications space. This environment leveraged the OpenAI Gym framework and featured customizable simulation scenarios within the RF spectrum. However, the RFRL Gym was limited to training a single RL agent per simulation; this is not ideal, as most real-world RF scenarios will contain multiple intelligent agents in cooperative, competitive, or mixed settings, which is a natural consequence of spectrum congestion. Therefore, through integration with Ray RLlib, multi-agent reinforcement learning (MARL) functionality for training and assessment has been added to the RFRL Gym, making it even more of a robust tool for RF spectrum simulation. This paper provides an overview of the updated RFRL Gym environment. In this work, the general framework of the tool is described relative to comparable existing resources, highlighting the significant additions and refactoring we have applied to the Gym. Afterward, results from testing various RF scenarios in the MARL environment and future additions are discussed.