Events as Triggers for Behavioral Diversity in Multi-Agent Reinforcement Learning

TL;DR

This paper introduces an event-driven framework for multi-agent reinforcement learning that enables dynamic behavioral diversity and role switching, improving task performance and generalization.

Contribution

It proposes a novel approach decoupling agent identity from behavior using events, Neural Manifold Diversity, and hypernetworks for on-the-fly policy reconfiguration.

Findings

Outperforms baselines on multiple benchmarks.

Achieves zero-shot generalization.

Solves tasks requiring sequential behavior reassignment.

Abstract

Effective multi-agent cooperation requires agents to adopt diverse behaviors as task conditions evolve-and to do so at the right moment. Yet, current Multi-Agent Reinforcement Learning (MARL) frameworks that facilitate this diversity are still limited by the fact that they bind fixed behaviors to fixed agent identities. Consequently, they are ill-equipped for tasks where agents need to take on different roles at very specific moments in time. We argue that, to define these behavioral transitions, the missing ingredient is $\textbf{events}$. Events are changes in the state of the system that induce qualitative changes in the task. Based on this view, we introduce a framework that decouples agent identity from behavior, capturing a continuous manifold from which agents instantiate their behaviors in response to events. This framework is based on two elements. First, to build an expressive behavior manifold, we introduce Neural Manifold Diversity (NMD), a formal distance metric that remains well-defined when behaviors are transient and agent-agnostic. Second, we use an event-based hypernetwork that generates Low-Rank Adaptation (LoRA) modules over a shared team policy, enabling on-the-fly agent-policy reconfiguration in response to events. We prove that this construction ensures that diversity does not interfere with reward maximization by design. Empirical results demonstrate that our framework outperforms established baselines across benchmarks while exhibiting zero-shot generalization, and being the only method that solves tasks requiring sequential behavior reassignment.