Communicating Plans, Not Percepts: Scalable Multi-Agent Coordination with Embodied World Models

TL;DR

This paper compares learned and engineered communication strategies in multi-agent systems, demonstrating that structured, world model-based communication outperforms end-to-end learned protocols in complex, goal-directed tasks.

Contribution

It introduces a novel engineered communication approach using embodied world models, showing its advantages over learned protocols in scalability and efficiency.

Findings

World model-based communication outperforms learned protocols in complex environments.

Engineered structured communication improves sample efficiency.

Structured models enable more scalable multi-agent coordination.

Abstract

Robust coordination is critical for effective decision-making in multi-agent systems, especially under partial observability. A central question in Multi-Agent Reinforcement Learning (MARL) is whether to engineer communication protocols or learn them end-to-end. We investigate this dichotomy using embodied world models. We propose and compare two communication strategies for a cooperative task-allocation problem. The first, Learned Direct Communication (LDC), learns a protocol end-to-end. The second, Intention Communication, uses an engineered inductive bias: a compact, learned world model, the Imagined Trajectory Generation Module (ITGM), which uses the agent's own policy to simulate future states. A Message Generation Network (MGN) then compresses this plan into a message. We evaluate these approaches on goal-directed interaction in a grid world, a canonical abstraction for embodied AI problems, while scaling environmental complexity. Our experiments reveal that while emergent communication is viable in simple settings, the engineered, world model-based approach shows superior performance, sample efficiency, and scalability as complexity increases. These findings advocate for integrating structured, predictive models into MARL agents to enable active, goal-driven coordination.