Planning Like Human: A Dual-process Framework for Dialogue Planning

TL;DR

This paper introduces a dual-process framework for dialogue planning inspired by psychology, combining fast instinctive responses and slow deliberative strategies to improve goal-oriented conversations in large language models.

Contribution

It proposes the DPDP framework with two planning systems and a novel training regimen, enhancing dialogue quality and efficiency over existing methods.

Findings

DPDP outperforms existing methods in dialogue quality.

The dual-process approach balances efficiency and strategic depth.

Empirical results show improved goal achievement in dialogues.

Abstract

In proactive dialogue, the challenge lies not just in generating responses but in steering conversations toward predetermined goals, a task where Large Language Models (LLMs) typically struggle due to their reactive nature. Traditional approaches to enhance dialogue planning in LLMs, ranging from elaborate prompt engineering to the integration of policy networks, either face efficiency issues or deliver suboptimal performance. Inspired by the dualprocess theory in psychology, which identifies two distinct modes of thinking - intuitive (fast) and analytical (slow), we propose the Dual-Process Dialogue Planning (DPDP) framework. DPDP embodies this theory through two complementary planning systems: an instinctive policy model for familiar contexts and a deliberative Monte Carlo Tree Search (MCTS) mechanism for complex, novel scenarios. This dual strategy is further coupled with a novel two-stage training regimen: offline Reinforcement Learning for robust initial policy model formation followed by MCTS-enhanced on-the-fly learning, which ensures a dynamic balance between efficiency and strategic depth. Our empirical evaluations across diverse dialogue tasks affirm DPDP's superiority in achieving both high-quality dialogues and operational efficiency, outpacing existing methods.