Structured Diversity Control: A Dual-Level Framework for Group-Aware Multi-Agent Coordination

TL;DR

Structured Diversity Control (SDC) introduces a dual-level, group-aware framework for regulating behavioral diversity in multi-agent reinforcement learning, leading to significant performance improvements in complex collaborative tasks.

Contribution

The paper proposes SDC, a novel framework that defines diversity as intra-group cohesion and inter-group specialization, enabling fine-grained, structural control without reward modification.

Findings

Increased average rewards by up to 47.1% in multi-target pursuit.

Reduced episode lengths by 12.82% in neutralization scenarios.

Provides a new analytical perspective on cooperation in multi-agent systems.

Abstract

Controlling the behavioral diversity is a pivotal challenge in multi-agent reinforcement learning (MARL), particularly in complex collaborative scenarios. While existing methods attempt to regulate behavioral diversity by directly differentiating across all agents, they lack deep characterization and learning of multi-agent composition structures. This limitation leads to suboptimal performance or coordination failures when facing more complex or challenging tasks. To bridge this gap, we introduce Structured Diversity Control (SDC), a framework that redefines the system-wide diversity metric as a weighted combination of intra-group diversity, which is minimized for cohesion and inter-group diversity, which is maximized for specialization. The trade-off is governed by a pre-set Diversity Structure Factor (DSF), allowing for fine-grained, group-aware control over the collective strategy. Our method directly constrains the policy architecture without altering reward functions. This structural definition of diversity enables SDC to deliver substantial performance gains across various experiments, including increasing average rewards by up to 47.1\% in multi-target pursuit and reducing episode lengths by 12.82\% in complex neutralization scenarios. The proposed method offers a novel analytical perspective on the problem of cooperation in group-aware multi-agent systems.