Safe Heterogeneous Multi-Agent RL with Communication Regularization for Coordinated Target Acquisition

TL;DR

This paper presents a decentralized multi-agent reinforcement learning framework that combines graph-based communication, safety filters, and structured rewards to enable heterogeneous agents to safely and effectively discover and acquire targets in complex environments.

Contribution

It introduces a novel integrated framework combining graph attention communication, safety filters, and structured rewards for heterogeneous multi-agent target acquisition tasks.

Findings

Effective target discovery and acquisition demonstrated in simulations

Enhanced safety and stability in multi-agent coordination

Structured reward promotes collision avoidance and informational orthogonality

Abstract

This paper introduces a decentralized multi-agent reinforcement learning framework enabling structurally heterogeneous teams of agents to jointly discover and acquire randomly located targets in environments characterized by partial observability, communication constraints, and dynamic interactions. Each agent's policy is trained with the Multi-Agent Proximal Policy Optimization algorithm and employs a Graph Attention Network encoder that integrates simulated range-sensing data with communication embeddings exchanged among neighboring agents, enabling context-aware decision-making from both local sensing and relational information. In particular, this work introduces a unified framework that integrates graph-based communication and trajectory-aware safety through safety filters. The architecture is supported by a structured reward formulation designed to encourage effective target discovery and acquisition, collision avoidance, and de-correlation between the agents' communication vectors by promoting informational orthogonality. The effectiveness of the proposed reward function is demonstrated through a comprehensive ablation study. Moreover, simulation results demonstrate safe and stable task execution, confirming the framework's effectiveness.