Iroko: A Framework to Prototype Reinforcement Learning for Data Center Traffic Control

TL;DR

Iroko is an open-source framework that enables rapid prototyping and evaluation of reinforcement learning-based congestion control algorithms in data center networks, addressing stability and overfitting issues.

Contribution

The paper introduces Iroko, a new emulator supporting diverse network scenarios and RL algorithms, facilitating research in RL-based data center traffic control.

Findings

RL algorithms outperform TCP New Vegas on certain topologies

Iroko supports fast, fair evaluation of CC algorithms

Open-source availability encourages further research

Abstract

Recent networking research has identified that data-driven congestion control (CC) can be more efficient than traditional CC in TCP. Deep reinforcement learning (RL), in particular, has the potential to learn optimal network policies. However, RL suffers from instability and over-fitting, deficiencies which so far render it unacceptable for use in datacenter networks. In this paper, we analyze the requirements for RL to succeed in the datacenter context. We present a new emulator, Iroko, which we developed to support different network topologies, congestion control algorithms, and deployment scenarios. Iroko interfaces with the OpenAI gym toolkit, which allows for fast and fair evaluation of different RL and traditional CC algorithms under the same conditions. We present initial benchmarks on three deep RL algorithms compared to TCP New Vegas and DCTCP. Our results show that these algorithms are able to learn a CC policy which exceeds the performance of TCP New Vegas on a dumbbell and fat-tree topology. We make our emulator open-source and publicly available: https://github.com/dcgym/iroko