MACS: Deep Reinforcement Learning based SDN Controller Synchronization Policy Design

TL;DR

This paper introduces MACS, a deep reinforcement learning-based policy for SDN controller synchronization that significantly improves network performance by learning optimal synchronization strategies.

Contribution

It formulates controller synchronization as an MDP and develops MACS, a novel RL-based policy that outperforms existing heuristics in SDN environments.

Findings

MACS achieves 56% performance improvement over ONOS.

MACS outperforms greedy heuristics by 30%.

Deep neural networks effectively capture environment patterns.

Abstract

In distributed software-defined networks (SDN), multiple physical SDN controllers, each managing a network domain, are implemented to balance centralised control, scalability, and reliability requirements. In such networking paradigms, controllers synchronize with each other, in attempts to maintain a logically centralised network view. Despite the presence of various design proposals for distributed SDN controller architectures, most existing works only aim at eliminating anomalies arising from the inconsistencies in different controllers' network views. However, the performance aspect of controller synchronization designs with respect to given SDN applications are generally missing. To fill this gap, we formulate the controller synchronization problem as a Markov decision process (MDP) and apply reinforcement learning techniques combined with deep neural networks (DNNs) to train a smart, scalable, and fine-grained controller synchronization policy, called the Multi-Armed Cooperative Synchronization (MACS), whose goal is to maximise the performance enhancements brought by controller synchronizations. Evaluation results confirm the DNN's exceptional ability in abstracting latent patterns in the distributed SDN environment, rendering significant superiority to MACS-based synchronization policy, which are 56% and 30% performance improvements over ONOS and greedy SDN controller synchronization heuristics.