An Overlay Multicast Routing Method Based on Network Situational Awareness and Hierarchical Multi-Agent Reinforcement Learning

TL;DR

This paper introduces MA-DHRL-OM, a hierarchical reinforcement learning-based overlay multicast routing method that improves adaptability, stability, and performance in dynamic networks by leveraging SDN's global view.

Contribution

It presents a novel multi-agent deep hierarchical reinforcement learning approach that decomposes multicast routing into stages, enhancing scalability and convergence stability.

Findings

Outperforms existing methods in delay, bandwidth utilization, and packet loss.

Achieves more stable convergence and flexible routing.

Utilizes SDN's global view for traffic-aware path planning.

Abstract

Compared with IP multicast, Overlay Multicast (OM) offers better compatibility and flexible deployment in heterogeneous, cross-domain networks. However, traditional OM struggles to adapt to dynamic traffic due to unawareness of physical resource states, and existing reinforcement learning methods fail to decouple OM's tightly coupled multi-objective nature, leading to high complexity, slow convergence, and instability. To address this, we propose MA-DHRL-OM, a multi-agent deep hierarchical reinforcement learning approach. Using SDN's global view, it builds a traffic-aware model for OM path planning. The method decomposes OM tree construction into two stages via hierarchical agents, reducing action space and improving convergence stability. Multi-agent collaboration balances multi-objective optimization while enhancing scalability and adaptability. Experiments show MA-DHRL-OM outperforms existing methods in delay, bandwidth utilization, and packet loss, with more stable convergence and flexible routing.