ns3-gym: Extending OpenAI Gym for Networking Research

TL;DR

The paper introduces ns3-gym, a framework that integrates OpenAI Gym with the ns-3 network simulator to facilitate reinforcement learning research in networking, providing a flexible, open-source tool for the community.

Contribution

It presents the design and implementation of ns3-gym, enabling RL experiments within ns-3, which was previously lacking an integrated RL framework.

Findings

Framework successfully integrates OpenAI Gym with ns-3

Two illustrative RL examples implemented using ns3-gym

Open source release under GPL license

Abstract

OpenAI Gym is a toolkit for reinforcement learning (RL) research. It includes a large number of well-known problems that expose a common interface allowing to directly compare the performance results of different RL algorithms. Since many years, the ns-3 network simulation tool is the de-facto standard for academic and industry research into networking protocols and communications technology. Numerous scientific papers were written reporting results obtained using ns-3, and hundreds of models and modules were written and contributed to the ns-3 code base. Today as a major trend in network research we see the use of machine learning tools like RL. What is missing is the integration of a RL framework like OpenAI Gym into the network simulator ns-3. This paper presents the ns3-gym framework. First, we discuss design decisions that went into the software. Second, two illustrative examples implemented using ns3-gym are presented. Our software package is provided to the community as open source under a GPL license and hence can be easily extended.