Towards Real-Time Routing Optimization with Deep Reinforcement Learning: Open Challenges

TL;DR

This paper explores the potential of deep reinforcement learning for real-time network routing optimization, highlighting current challenges and the need for further research to develop practical, production-ready solutions.

Contribution

It reviews recent DRL advances for network routing and discusses open challenges to enable real-time, adaptive, and scalable DRL-based network control systems.

Findings

DRL shows promise for online routing optimization.

Current DRL methods face open challenges for deployment.

Further research needed for production-ready solutions.

Abstract

The digital transformation is pushing the existing network technologies towards new horizons, enabling new applications (e.g., vehicular networks). As a result, the networking community has seen a noticeable increase in the requirements of emerging network applications. One main open challenge is the need to accommodate control systems to highly dynamic network scenarios. Nowadays, existing network optimization technologies do not meet the needed requirements to effectively operate in real time. Some of them are based on hand-crafted heuristics with limited performance and adaptability, while some technologies use optimizers which are often too time-consuming. Recent advances in Deep Reinforcement Learning (DRL) have shown a dramatic improvement in decision-making and automated control problems. Consequently, DRL represents a promising technique to efficiently solve a variety of relevant network optimization problems, such as online routing. In this paper, we explore the use of state-of-the-art DRL technologies for real-time routing optimization and outline some relevant open challenges to achieve production-ready DRL-based solutions.