Graph Attention Reinforcement Learning for Multicast Routing and Age-Optimal Scheduling

TL;DR

This paper introduces a novel reinforcement learning framework with graph attention mechanisms to optimize multicast routing and scheduling, significantly reducing Age of Information and improving computational efficiency in dynamic networks.

Contribution

It presents the first RL-based approach for multicast routing using graph embedding with NGAT, and jointly optimizes routing and scheduling for age minimization.

Findings

Achieves up to 9.85x computational efficiency over traditional algorithms.

Attains approximation ratios of 1.1-1.3, comparable to SOTA methods.

Reduces average weighted AoI by 25.6% and peak age by 29.2% in experiments.

Abstract

Multicast routing is essential for real-time group applications, such as video streaming, virtual reality, and metaverse platforms, where the Age of Information (AoI) acts as a crucial metric to assess information timeliness. This paper studies dynamic multicast networks with the objective of minimizing the expected average Age of Information (AoI) by jointly optimizing multicast routing and scheduling. The main challenges stem from the intricate coupling between routing and scheduling decisions, the inherent complexity of multicast operations, and the graph representation. We first decompose the original problem into two subtasks amenable to hierarchical reinforcement learning (RL) methods. We propose the first RL framework to address the multicast routing problem, also known as the Steiner Tree problem, by incorporating graph embedding and the successive addition of nodes and links. For graph embedding, we propose the Normalized Graph Attention mechanism (NGAT) framework with a proven contraction mapping property, enabling effective graph information capture and superior generalization within the hierarchical RL framework. We validate our framework through experiments on four datasets, including the real-world AS-733 dataset. The results demonstrate that our proposed scheme can be up to 9.85 times more computationally efficient than traditional multicast routing algorithms, achieving approximation ratios of 1.1-1.3 that are not only comparable to state-of-the-art (SOTA) methods but also highlight its superior generalization capabilities, performing effectively on unseen and more complex tasks. Additionally, our age-optimal TGMS algorithm reduces the average weighted Age of Information (AoI) by 25.6% and the weighted peak age by 29.2% under low-energy scenarios.