A Heuristically Assisted Deep Reinforcement Learning Approach for Network Slice Placement

TL;DR

This paper presents a hybrid deep reinforcement learning method combined with heuristics for efficient network slice placement, improving learning speed and resource utilization over existing approaches.

Contribution

It introduces a novel Heuristically-Assisted DRL framework using A3C and GCNs for optimized network slice placement automation.

Findings

Accelerated learning process demonstrated.

Improved resource efficiency shown.

Outperforms state-of-the-art methods.

Abstract

Network Slice placement with the problem of allocation of resources from a virtualized substrate network is an optimization problem which can be formulated as a multiobjective Integer Linear Programming (ILP) problem. However, to cope with the complexity of such a continuous task and seeking for optimality and automation, the use of Machine Learning (ML) techniques appear as a promising approach. We introduce a hybrid placement solution based on Deep Reinforcement Learning (DRL) and a dedicated optimization heuristic based on the Power of Two Choices principle. The DRL algorithm uses the so-called Asynchronous Advantage Actor Critic (A3C) algorithm for fast learning, and Graph Convolutional Networks (GCN) to automate feature extraction from the physical substrate network. The proposed Heuristically-Assisted DRL (HA-DRL) allows to accelerate the learning process and gain in resource usage when compared against other state-of-the-art approaches as the evaluation results evidence.