ChatSOP: An SOP-Guided MCTS Planning Framework for Controllable LLM Dialogue Agents

TL;DR

This paper introduces ChatSOP, a framework that uses SOP-guided MCTS planning to improve the controllability of LLM-based dialogue agents, leading to more focused and effective conversations.

Contribution

It presents a novel SOP-guided MCTS framework, a curated SOP-annotated dialogue dataset, and a new method combining Chain of Thought reasoning with supervised fine-tuning for SOP prediction.

Findings

Achieved 27.95% improvement in action accuracy over GPT-3.5 baseline.

Demonstrated effectiveness on multiple models, including open-source variants.

Validated the approach with publicly available dataset and code.

Abstract

Dialogue agents powered by Large Language Models (LLMs) show superior performance in various tasks. Despite the better user understanding and human-like responses, their lack of controllability remains a key challenge, often leading to unfocused conversations or task failure. To address this, we introduce Standard Operating Procedure (SOP) to regulate dialogue flow. Specifically, we propose ChatSOP, a novel SOP-guided Monte Carlo Tree Search (MCTS) planning framework designed to enhance the controllability of LLM-driven dialogue agents. To enable this, we curate a dataset comprising SOP-annotated multi-scenario dialogues, generated using a semi-automated role-playing system with GPT-4o and validated through strict manual quality control. Additionally, we propose a novel method that integrates Chain of Thought reasoning with supervised fine-tuning for SOP prediction and utilizes SOP-guided Monte Carlo Tree Search for optimal action planning during dialogues. Experimental results demonstrate the effectiveness of our method, such as achieving a 27.95% improvement in action accuracy compared to baseline models based on GPT-3.5 and also showing notable gains for open-source models. Dataset and codes are publicly available.