Learn as Individuals, Evolve as a Team: Multi-agent LLMs Adaptation in Embodied Environments

TL;DR

This paper introduces LIET, a framework enabling multi-agent LLMs to learn individually and evolve collaboratively during testing, significantly improving their adaptation and cooperation in embodied environments.

Contribution

The paper proposes a novel LIET paradigm that combines individual learning and team evolution for multi-agent LLM adaptation in embodied scenarios.

Findings

LIET outperforms existing baselines on benchmark tasks.

Enhanced cooperative planning abilities demonstrated with LLaMA and GPT-4o.

Effective environment understanding through local utility functions.

Abstract

Large language models (LLMs) possess extensive knowledge bases and strong reasoning capabilities, making them promising tools for complex, multi-agent planning in embodied environments. However, despite LLMs' advanced abilities and the sophisticated modular design of agentic methods, existing LLM-based planning algorithms remain limited by weak adaptation capabilities to multi-agent embodied scenarios. We address this limitation by introducing a framework that enables LLM agents to learn and evolve both before and during test time, equipping them with environment-relevant knowledge for better planning and enhanced communication for improved cooperation. Inspired by centralized training with decentralized execution in multi-agent reinforcement learning, we propose a \textit{Learn as Individuals, Evolve as a Team (LIET)} paradigm for multi-agent LLMs adaptation. At the individual level, LLM agents learn a local utility function from exploratory datasets to better comprehend the embodied environment, which is then queried during test time to support informed decision-making. At the team level, LLM agents collaboratively and iteratively maintain and update a shared cooperation knowledge list based on new experiences, using it to guide more effective communication. By combining individual learning with team evolution, LIET enables comprehensive and flexible adaptation for LLM agents. Our experiments on Communicative Watch-And-Help and ThreeD-World Multi-Agent Transport benchmarks demonstrate that LIET, instantiated with both LLaMA and GPT-4o, outperforms existing baselines and exhibits strong cooperative planning abilities.