Near Optimal VNF Placement in Edge-Enabled 6G Networks

TL;DR

This paper addresses the challenge of optimal Virtual Network Function placement in 6G edge networks, proposing a theoretical bound and a practical reinforcement learning solution that performs near-optimally under resource constraints.

Contribution

It introduces a theoretical performance bound for VNF placement using Markov Decision Processes and develops a Q-Learning based practical solution for efficient resource management in 6G edge networks.

Findings

Q-Learning achieves near-optimal performance compared to the theoretical bound.

The approach effectively manages scarce edge resources.

Simulation results validate the practicality of the proposed method.

Abstract

Softwarization and virtualization are key concepts for emerging industries that require ultra-low latency. This is only possible if computing resources, traditionally centralized at the core of communication networks, are moved closer to the user, to the network edge. However, the realization of Edge Computing (EC) in the sixth generation (6G) of mobile networks requires efficient resource allocation mechanisms for the placement of the Virtual Network Functions (VNFs). Machine learning (ML) methods, and more specifically, Reinforcement Learning (RL), are a promising approach to solve this problem. The main contributions of this work are twofold: first, we obtain the theoretical performance bound for VNF placement in EC-enabled6G networks by formulating the problem mathematically as a finite Markov Decision Process (MDP) and solving it using a dynamic programming method called Policy Iteration (PI). Second, we develop a practical solution to the problem using RL, where the problem is treated with Q-Learning that considers both computational and communication resources when placing VNFs in the network. The simulation results under different settings of the system parameters show that the performance of the Q-Learning approach is close to the optimal PI algorithm (without having its restrictive assumptions on service statistics). This is particularly interesting when the EC resources are scarce and efficient management of these resources is required.