On the Robustness of Controlled Deep Reinforcement Learning for Slice Placement

TL;DR

This paper evaluates the robustness of deep reinforcement learning algorithms for network slice placement under non-stationary traffic loads, demonstrating that hybrid DRL-heuristic methods outperform pure DRL in unpredictable network conditions.

Contribution

It introduces a hybrid DRL-heuristic approach for network slice placement and demonstrates its superior robustness over pure DRL in dynamic, unpredictable network environments.

Findings

Hybrid DRL-heuristic reduces performance degradation under load variations.

Pure DRL is less robust to traffic unpredictability.

Hybrid approach is more suitable for real network scenarios.

Abstract

The evaluation of the impact of using Machine Learning in the management of softwarized networks is considered in multiple research works. Beyond that, we propose to evaluate the robustness of online learning for optimal network slice placement. A major assumption to this study is to consider that slice request arrivals are non-stationary. In this context, we simulate unpredictable network load variations and compare two Deep Reinforcement Learning (DRL) algorithms: a pure DRL-based algorithm and a heuristically controlled DRL as a hybrid DRL-heuristic algorithm, to assess the impact of these unpredictable changes of traffic load on the algorithms performance. We conduct extensive simulations of a large-scale operator infrastructure. The evaluation results show that the proposed hybrid DRL-heuristic approach is more robust and reliable in case of unpredictable network load changes than pure DRL as it reduces the performance degradation. These results are follow-ups for a series of recent research we have performed showing that the proposed hybrid DRL-heuristic approach is efficient and more adapted to real network scenarios than pure DRL.