PLAYER*: Enhancing LLM-based Multi-Agent Communication and Interaction in Murder Mystery Games

TL;DR

This paper introduces WellPlay, a comprehensive dataset for reasoning in Murder Mystery Games, and presents PLAYER*, a novel LLM-based framework that improves multi-agent reasoning and interaction in complex social scenarios.

Contribution

The paper provides a new dataset for social reasoning in MMGs and proposes PLAYER*, a framework enhancing LLM-based agent reasoning and interaction capabilities.

Findings

PLAYER* outperforms existing methods in reasoning accuracy

PLAYER* improves efficiency in agent decision-making

PLAYER* enhances agent-human interaction quality

Abstract

We introduce WellPlay, a reasoning dataset for multi-agent conversational inference in Murder Mystery Games (MMGs). WellPlay comprises 1,482 inferential questions across 12 games, spanning objectives, reasoning, and relationship understanding, and establishes a systematic benchmark for evaluating agent reasoning abilities in complex social settings. Building on this foundation, we present PLAYER*, a novel framework for Large Language Model (LLM)-based agents in MMGs. MMGs pose unique challenges, including undefined state spaces, absent intermediate rewards, and the need for strategic reasoning through natural language. PLAYER* addresses these challenges with a sensor-based state representation and an information-driven strategy that optimises questioning and suspect pruning. Experiments show that PLAYER* outperforms existing methods in reasoning accuracy, efficiency, and agent-human interaction, advancing reasoning agents for complex social scenarios.