Prompt-Based Monte-Carlo Tree Search for Goal-Oriented Dialogue Policy Planning

TL;DR

GDP-Zero leverages open-loop Monte Carlo Tree Search with large language models to plan goal-oriented dialogues without training, achieving higher persuasion success than ChatGPT.

Contribution

This paper introduces GDP-Zero, a novel approach that uses LLMs as a policy prior, value function, and models within MCTS, eliminating the need for training data.

Findings

Responses preferred over ChatGPT up to 59.32% of the time

Rated more persuasive than ChatGPT in interactive evaluations

Effective goal-oriented dialogue planning without model training

Abstract

Planning for goal-oriented dialogue often requires simulating future dialogue interactions and estimating task progress. Many approaches thus consider training neural networks to perform look-ahead search algorithms such as A* search and Monte Carlo Tree Search (MCTS). However, this training often requires abundant annotated data, which creates challenges when faced with noisy annotations or low-resource settings. We introduce GDP-Zero, an approach using Open-Loop MCTS to perform goal-oriented dialogue policy planning without any model training. GDP-Zero prompts a large language model to act as a policy prior, value function, user simulator, and system model during the tree search. We evaluate GDP-Zero on the goal-oriented task PersuasionForGood, and find that its responses are preferred over ChatGPT up to 59.32% of the time, and are rated more persuasive than ChatGPT during interactive evaluations.