Communication and Verification in LLM Agents towards Collaboration under Information Asymmetry

TL;DR

This paper explores how LLM agents can collaborate effectively under information asymmetry by studying communication, verification, and reasoning strategies within a symbolic puzzle game, improving understanding and safety.

Contribution

It introduces a framework for LLM agent collaboration with communication and verification mechanisms, extending Einstein Puzzles to a multi-agent tabletop game.

Findings

Aligned communication improves collaboration effectiveness

Agents without communication lack true rule understanding

Environment-based verification enhances safety and interpretability

Abstract

While Large Language Model (LLM) agents are often approached from the angle of action planning/generation to accomplish a goal (e.g., given by language descriptions), their abilities to collaborate with each other to achieve a joint goal are not well explored. To address this limitation, this paper studies LLM agents in task collaboration, particularly under the condition of information asymmetry, where agents have disparities in their knowledge and skills and need to work together to complete a shared task. We extend Einstein Puzzles, a classical symbolic puzzle, to a table-top game. In this game, two LLM agents must reason, communicate, and act to satisfy spatial and relational constraints required to solve the puzzle. We apply a fine-tuning-plus-verifier framework in which LLM agents are equipped with various communication strategies and verification signals from the environment. Empirical results highlight the critical importance of aligned communication, especially when agents possess both information-seeking and -providing capabilities. Interestingly, agents without communication can still achieve high task performance; however, further analysis reveals a lack of true rule understanding and lower trust from human evaluators. Instead, by integrating an environment-based verifier, we enhance agents' ability to comprehend task rules and complete tasks, promoting both safer and more interpretable collaboration in AI systems. https://github.com/Roihn/EinsteinPuzzles