HYGMA: Hypergraph Coordination Networks with Dynamic Grouping for Multi-Agent Reinforcement Learning

TL;DR

HYGMA introduces a dynamic hypergraph neural network framework with spectral clustering and attention mechanisms to improve coordination and information exchange in multi-agent reinforcement learning, leading to superior performance.

Contribution

It presents a novel adaptive hypergraph construction method that captures higher-order agent relationships for enhanced multi-agent coordination.

Findings

Outperforms state-of-the-art methods in cooperative tasks

Improves sample efficiency and final performance

Effectively models complex agent relationships

Abstract

Cooperative multi-agent reinforcement learning faces significant challenges in effectively organizing agent relationships and facilitating information exchange, particularly when agents need to adapt their coordination patterns dynamically. This paper presents a novel framework that integrates dynamic spectral clustering with hypergraph neural networks to enable adaptive group formation and efficient information processing in multi-agent systems. The proposed framework dynamically constructs and updates hypergraph structures through spectral clustering on agents' state histories, enabling higher-order relationships to emerge naturally from agent interactions. The hypergraph structure is enhanced with attention mechanisms for selective information processing, providing an expressive and efficient way to model complex agent relationships. This architecture can be implemented in both value-based and policy-based paradigms through a unified objective combining task performance with structural regularization. Extensive experiments on challenging cooperative tasks demonstrate that our method significantly outperforms state-of-the-art approaches in both sample efficiency and final performance.