DVM: Towards Controllable LLM Agents in Social Deduction Games

TL;DR

This paper introduces DVM, a framework that enables large language model agents to adapt their skill levels in social deduction games like Werewolf, balancing gameplay and ensuring safety and fairness.

Contribution

DVM is a novel framework that combines reinforcement learning with a win rate-constrained decision mechanism to control LLM agent proficiency in social deduction games.

Findings

DVM outperforms existing methods in Werewolf.

DVM can modulate performance to meet target win rates.

The framework enhances adaptive and balanced gameplay.

Abstract

Large Language Models (LLMs) have advanced the capability of game agents in social deduction games (SDGs). These games rely heavily on conversation-driven interactions and require agents to infer, make decisions, and express based on such information. While this progress leads to more sophisticated and strategic non-player characters (NPCs) in SDGs, there exists a need to control the proficiency of these agents. This control not only ensures that NPCs can adapt to varying difficulty levels during gameplay, but also provides insights into the safety and fairness of LLM agents. In this paper, we present DVM, a novel framework for developing controllable LLM agents for SDGs, and demonstrate its implementation on one of the most popular SDGs, Werewolf. DVM comprises three main components: Predictor, Decider, and Discussor. By integrating reinforcement learning with a win rate-constrained decision chain reward mechanism, we enable agents to dynamically adjust their gameplay proficiency to achieve specified win rates. Experiments show that DVM not only outperforms existing methods in the Werewolf game, but also successfully modulates its performance levels to meet predefined win rate targets. These results pave the way for LLM agents' adaptive and balanced gameplay in SDGs, opening new avenues for research in controllable game agents.