Actor-Critic-Based Learning for Zero-touch Joint Resource and Energy Control in Network Slicing

TL;DR

This paper introduces a novel Actor-Critic-based deep reinforcement learning approach for zero-touch network slicing management, aiming to optimize energy use and reduce costs in 5G and beyond networks.

Contribution

It proposes a twin-delayed double-Q soft Actor-Critic method for stable learning in network slicing, integrating AI into a knowledge plane framework for automated network management.

Findings

Reduces energy consumption in network slicing

Decreases virtual network function instantiation costs

Improves time efficiency and CPU utilization

Abstract

To harness the full potential of beyond 5G (B5G) communication systems, zero-touch network slicing (NS) is viewed as a promising fully-automated management and orchestration (MANO) system. This paper proposes a novel knowledge plane (KP)-based MANO framework that accommodates and exploits recent NS technologies and is termed KB5G. Specifically, we deliberate on algorithmic innovation and artificial intelligence (AI) in KB5G. We invoke a continuous model-free deep reinforcement learning (DRL) method to minimize energy consumption and virtual network function (VNF) instantiation cost. We present a novel Actor-Critic-based NS approach to stabilize learning called, twin-delayed double-Q soft Actor-Critic (TDSAC) method. The TDSAC enables central unit (CU) to learn continuously to accumulate the knowledge learned in the past to minimize future NS costs. Finally, we present numerical results to showcase the gain of the adopted approach and verify the performance in terms of energy consumption, CPU utilization, and time efficiency.