DRL-based Slice Placement Under Non-Stationary Conditions

TL;DR

This paper introduces hybrid deep reinforcement learning algorithms for dynamic network slice placement under non-stationary request patterns, demonstrating faster convergence and higher reliability than pure DRL methods in large-scale simulations.

Contribution

The paper proposes novel hybrid DRL-heuristic algorithms that outperform pure DRL in non-stationary network environments, reducing learning episodes significantly.

Findings

Hybrid algorithms require three orders of magnitude fewer episodes to converge.

Hybrid DRL-heuristic methods are more reliable in non-stationary scenarios.

Extensive simulations validate the effectiveness of the proposed approaches.

Abstract

We consider online learning for optimal network slice placement under the assumption that slice requests arrive according to a non-stationary Poisson process. We propose a framework based on Deep Reinforcement Learning (DRL) combined with a heuristic to design algorithms. We specifically design two pure-DRL algorithms and two families of hybrid DRL-heuristic algorithms. To validate their performance, we perform extensive simulations in the context of a large-scale operator infrastructure. The evaluation results show that the proposed hybrid DRL-heuristic algorithms require three orders of magnitude of learning episodes less than pure-DRL to achieve convergence. This result indicates that the proposed hybrid DRL-heuristic approach is more reliable than pure-DRL in a real non-stationary network scenario.