Dynamic Graph Communication for Decentralised Multi-Agent Reinforcement Learning

TL;DR

This paper introduces a dynamic graph communication framework for decentralized multi-agent reinforcement learning, improving decision-making and routing performance in dynamic networks with node failures.

Contribution

It adapts static network routing to dynamic environments, integrating graph attention networks and multi-round communication for better coordination.

Findings

9.5% increase in average rewards

6.4% reduction in communication overhead

Effective training of attention-based aggregation in sparse-reward settings

Abstract

This work presents a novel communication framework for decentralized multi-agent systems operating in dynamic network environments. Integrated into a multi-agent reinforcement learning system, the framework is designed to enhance decision-making by optimizing the network's collective knowledge through efficient communication. Key contributions include adapting a static network packet-routing scenario to a dynamic setting with node failures, incorporating a graph attention network layer in a recurrent message-passing framework, and introducing a multi-round communication targeting mechanism. This approach enables an attention-based aggregation mechanism to be successfully trained within a sparse-reward, dynamic network packet-routing environment using only reinforcement learning. Experimental results show improvements in routing performance, including a 9.5 percent increase in average rewards and a 6.4 percent reduction in communication overhead compared to a baseline system. The study also examines the ethical and legal implications of deploying such systems in critical infrastructure and military contexts, identifies current limitations, and suggests potential directions for future research.