Scalability Assurance in SFC provisioning via Distributed Design for Deep Reinforcement Learning

TL;DR

This paper introduces a distributed deep reinforcement learning framework for scalable service function chaining provisioning, significantly improving acceptance ratios and reducing delays in large-scale networks.

Contribution

It proposes a novel distributed DRL-based framework dividing networks into clusters with local agents, enhancing scalability and efficiency over centralized solutions.

Findings

Up to 60% increase in service request acceptance ratio

Reduced end-to-end delay for accepted requests

Effective handling of large-scale network demands

Abstract

High-quality Service Function Chaining (SFC) provisioning is provided by the timely execution of Virtual Network Functions (VNFs) in a defined sequence. Advanced Deep Reinforcement Learning (DRL) solutions are utilized in many studies to contribute to fast and reliable autonomous SFC provisioning. However, under a large-scale network environment, centralized solutions might struggle to provide efficient outcomes when handling massive demands with stringent End-to-End (E2E) delay constraints. Therefore, in this paper, a novel distributed SFC provisioning framework is proposed, where the network is divided into several clusters. Each cluster has a dedicated local agent with a DRL module to handle the SFC provisioning of demands in that cluster. Also, there is a general agent that can communicate with local agents to handle the requests beyond their capacity. The DRL module of local agents can be applied under different configurations of clusters independent of different numbers of data centers and logical links in each cluster. Simulation results demonstrate that utilizing the proposed distributed framework offers up to 60% improvements in the acceptance ratio of service requests in comparison to the centralized approach while minimizing the E2E delay of accepted requests.