Language-Driven Coordination and Learning in Multi-Agent Simulation Environments

TL;DR

This paper presents LLM-MARL, a framework integrating large language models into multi-agent reinforcement learning to improve coordination, communication, and generalization in simulated environments, demonstrating significant performance gains.

Contribution

It introduces a novel modular framework that combines LLMs with MARL, enabling dynamic subgoal generation, symbolic messaging, and episodic memory for better multi-agent cooperation.

Findings

Improved win rates and coordination scores over baseline methods.

Significant zero-shot generalization in complex environments.

Emergent behaviors like role specialization and strategic communication.

Abstract

This paper introduces LLM-MARL, a unified framework that incorporates large language models (LLMs) into multi-agent reinforcement learning (MARL) to enhance coordination, communication, and generalization in simulated game environments. The framework features three modular components of Coordinator, Communicator, and Memory, which dynamically generate subgoals, facilitate symbolic inter-agent messaging, and support episodic recall. Training combines PPO with a language-conditioned loss and LLM query gating. LLM-MARL is evaluated in Google Research Football, MAgent Battle, and StarCraft II. Results show consistent improvements over MAPPO and QMIX in win rate, coordination score, and zero-shot generalization. Ablation studies demonstrate that subgoal generation and language-based messaging each contribute significantly to performance gains. Qualitative analysis reveals emergent behaviors such as role specialization and communication-driven tactics. By bridging language modeling and policy learning, this work contributes to the design of intelligent, cooperative agents in interactive simulations. It offers a path forward for leveraging LLMs in multi-agent systems used for training, games, and human-AI collaboration.