The Greatest Teacher, Failure is: Using Reinforcement Learning for SFC Placement Based on Availability and Energy Consumption

TL;DR

This paper introduces a reinforcement learning-based approach for dynamic placement of service function chains in networks, optimizing for availability and energy efficiency amid network heterogeneity and dynamism.

Contribution

It proposes and compares two RL algorithms, A2C and PPO2, for SFC placement, demonstrating improved performance over greedy methods in simulations based on a real network topology.

Findings

PPO2 outperforms A2C and greedy approaches in acceptance rate and energy use.

A2C performs better with more network resources.

Reinforcement learning effectively manages SFC placement challenges.

Abstract

Software defined networking (SDN) and network functions virtualisation (NFV) are making networks programmable and consequently much more flexible and agile. To meet service level agreements, achieve greater utilisation of legacy networks, faster service deployment, and reduce expenditure, telecommunications operators are deploying increasingly complex service function chains (SFCs). Notwithstanding the benefits of SFCs, increasing heterogeneity and dynamism from the cloud to the edge introduces significant SFC placement challenges, not least adding or removing network functions while maintaining availability, quality of service, and minimising cost. In this paper, an availability- and energy-aware solution based on reinforcement learning (RL) is proposed for dynamic SFC placement. Two policy-aware RL algorithms, Advantage Actor-Critic (A2C) and Proximal Policy Optimisation (PPO2), are compared using simulations of a ground truth network topology based on the Rede Nacional de Ensino e Pesquisa (RNP) Network, Brazil's National Teaching and Research Network backbone. The simulation results showed that PPO2 generally outperformed A2C and a greedy approach both in terms of acceptance rate and energy consumption. A2C outperformed PPO2 only in the scenario where network servers had a greater number of computing resources.