RateRL: A Framework for Developing RL-Based Rate Adaptation Algorithms in ns-3

TL;DR

RateRL is a novel framework integrating ns-3 and RL libraries to facilitate the development, validation, and fair comparison of RL-based rate adaptation algorithms in Wi-Fi networks.

Contribution

It introduces the first comprehensive tool for implementing, testing, and comparing RL-based rate adaptation algorithms in wireless networks.

Findings

Enables reproducible evaluation of RL algorithms in ns-3

Supports fair comparison of different RL-based RA approaches

Facilitates research and development in Wi-Fi rate adaptation

Abstract

The increasing complexity of recent Wi-Fi amendments is making the use of traditional algorithms and heuristics unfeasible to address the Rate Adaptation (RA) problem. This is due to the large combination of configuration parameters along with the high variability of the wireless channel. Recently, several works have proposed the usage of Reinforcement Learning (RL) techniques to address the problem. However, the proposed solutions lack sufficient technical explanation. Also, the lack of standard frameworks enabling the reproducibility of results and the limited availability of source code, makes the fair comparison with state of the art approaches a challenge. This paper proposes a framework, named RateRL, that integrates state of the art libraries with the well-known Network Simulator 3 (ns-3) to enable the implementation and evaluation of RL-based RA algorithms. To the best of our knowledge, RateRL is the first tool available to assist researchers during the implementation, validation and evaluation phases of RL-based RA algorithms and enable the fair comparison between competing algorithms.