Adaptive Theory of Mind for LLM-based Multi-Agent Coordination

TL;DR

This paper introduces an adaptive Theory of Mind (A-ToM) approach for LLM-based multi-agent systems, enabling agents to align their ToM reasoning with partners for improved coordination across various tasks.

Contribution

The paper proposes an adaptive ToM mechanism that estimates and aligns with partner's ToM order, enhancing multi-agent coordination.

Findings

A-ToM improves coordination in multi-agent tasks.

Misaligned ToM orders can impair agent performance.

The approach generalizes beyond LLM-based agents.

Abstract

Theory of Mind (ToM) refers to the ability to reason about others' mental states, and higher-order ToM involves considering that others also possess their own ToM. Equipping large language model (LLM)-driven agents with ToM has long been considered to improve their coordination in multiagent collaborative tasks. However, we find that misaligned ToM orders-mismatches in the depth of ToM reasoning between agents-can lead to insufficient or excessive reasoning about others, thereby impairing their coordination. To address this issue, we design an adaptive ToM (A-ToM) agent, which can align in ToM orders with its partner. Based on prior interactions, the agent estimates the partner's likely ToM order and leverages this estimation to predict the partner's action, thereby facilitating behavioral coordination. We conduct empirical evaluations on four multi-agent coordination tasks: a repeated matrix game, two grid navigation tasks and an Overcooked task. The results validate our findings on ToM alignment and demonstrate the effectiveness of our A-ToM agent. Furthermore, we discuss the generalizability of our A-ToM to non-LLM-based agents, as well as what would diminish the importance of ToM alignment.