Adaptive Network Intervention for Complex Systems: A Hierarchical Graph Reinforcement Learning Approach

TL;DR

This paper presents a Hierarchical Graph Reinforcement Learning framework for effective network interventions in complex multi-agent systems, emphasizing social learning effects and managerial authority constraints to enhance governance outcomes.

Contribution

The paper introduces a novel HGRL framework for network governance that adapts to social learning levels and authority constraints, outperforming existing methods.

Findings

HGRL maintains cooperation under low social learning.

High social learning leads to sparser, chain-like networks.

Managerial authority is crucial to prevent system failures.

Abstract

Effective governance and steering of behavior in complex multi-agent systems (MAS) are essential for managing system-wide outcomes, particularly in environments where interactions are structured by dynamic networks. In many applications, the goal is to promote pro-social behavior among agents, where network structure plays a pivotal role in shaping these interactions. This paper introduces a Hierarchical Graph Reinforcement Learning (HGRL) framework that governs such systems through targeted interventions in the network structure. Operating within the constraints of limited managerial authority, the HGRL framework demonstrates superior performance across a range of environmental conditions, outperforming established baseline methods. Our findings highlight the critical influence of agent-to-agent learning (social learning) on system behavior: under low social learning, the HGRL manager preserves cooperation, forming robust core-periphery networks dominated by cooperators. In contrast, high social learning accelerates defection, leading to sparser, chain-like networks. Additionally, the study underscores the importance of the system manager's authority level in preventing system-wide failures, such as agent rebellion or collapse, positioning HGRL as a powerful tool for dynamic network-based governance.