Do LLM-derived graph priors improve multi-agent coordination?

TL;DR

This paper explores how large language models can generate graph priors to improve multi-agent reinforcement learning coordination, demonstrating effectiveness across various scenarios and model sizes.

Contribution

It introduces a method for using LLMs to generate coordination graph priors for MARL, enhancing adaptability and performance in multi-agent tasks.

Findings

LLM-derived graph priors improve MARL coordination.

Models as small as 1.5B parameters are effective for prior generation.

The approach outperforms traditional coordination methods on benchmark scenarios.

Abstract

Multi-agent reinforcement learning (MARL) is crucial for AI systems that operate collaboratively in distributed and adversarial settings, particularly in multi-domain operations (MDO). A central challenge in cooperative MARL is determining how agents should coordinate: existing approaches must either hand-specify graph topology, rely on proximity-based heuristics, or learn structure entirely from environment interaction; all of which are brittle, semantically uninformed, or data-intensive. We investigate whether large language models (LLMs) can generate useful coordination graph priors for MARL by using minimal natural language descriptions of agent observations to infer latent coordination patterns. These priors are integrated into MARL algorithms via graph convolutional layers within a graph neural network (GNN)-based pipeline, and evaluated on four cooperative scenarios from the Multi-Agent Particle Environment (MPE) benchmark against baselines spanning the full spectrum of coordination modeling, from independent learners to state-of-the-art graph-based methods. We further ablate across five compact open-source LLMs to assess the sensitivity of prior quality to model choice. Our results provide the first quantitative evidence that LLM-derived graph priors can enhance coordination and adaptability in dynamic multi-agent environments, and demonstrate that models as small as 1.5B parameters are sufficient for effective prior generation.