Theory of Mind for Multi-Agent Collaboration via Large Language Models

TL;DR

This paper investigates the capabilities of Large Language Models in multi-agent collaboration, demonstrating emergent ToM behaviors, identifying limitations, and proposing belief state representations to improve performance in cooperative tasks.

Contribution

It introduces a framework for evaluating LLMs in multi-agent ToM tasks and proposes explicit belief states to enhance their reasoning and planning abilities.

Findings

LLMs show emergent collaborative and ToM behaviors

Explicit belief states improve task performance and ToM accuracy

LLMs face challenges with long-horizon planning and hallucinations

Abstract

While Large Language Models (LLMs) have demonstrated impressive accomplishments in both reasoning and planning, their abilities in multi-agent collaborations remains largely unexplored. This study evaluates LLM-based agents in a multi-agent cooperative text game with Theory of Mind (ToM) inference tasks, comparing their performance with Multi-Agent Reinforcement Learning (MARL) and planning-based baselines. We observed evidence of emergent collaborative behaviors and high-order Theory of Mind capabilities among LLM-based agents. Our results reveal limitations in LLM-based agents' planning optimization due to systematic failures in managing long-horizon contexts and hallucination about the task state. We explore the use of explicit belief state representations to mitigate these issues, finding that it enhances task performance and the accuracy of ToM inferences for LLM-based agents.