Hierarchical Dynamic Routing in Complex Networks via Topologically-decoupled and Cooperative Reinforcement Learning Agents

TL;DR

This paper introduces a hierarchical reinforcement learning-based dynamic routing strategy for complex networks that improves transport capacity and resilience by deploying a small number of cooperative, topologically-decoupled agents.

Contribution

It proposes a novel hierarchical routing method using reinforcement learning agents at high-centrality nodes, decoupled and cooperative, to enhance network capacity and robustness.

Findings

Transport capacity significantly improved with few agents.

Effective in real-world Internet networks at router and autonomous system levels.

Strategy remains resilient to link removals.

Abstract

The transport capacity of a communication network can be characterized by the transition from a free-flow state to a congested state. Here, we propose a dynamic routing strategy in complex networks based on hierarchical bypass selections. The routing decisions are made by the reinforcement learning agents implemented at selected nodes with high betweenness centrality. The learning processes of the agents are decoupled from each other due to the degeneracy of their bypasses. Through interactions mediated by the underlying traffic dynamics, the agents act cooperatively, and coherent actions arise spontaneously. With only a small number of agents, the transport capacities are significantly improved, including in real-world Internet networks at the router level and the autonomous system level. Our strategy is also resilient to link removals.