Uncertainty as a Planning Signal: Multi-Turn Decision Making for Goal-Oriented Conversation

TL;DR

This paper introduces CUP, a framework that uses uncertainty as a planning signal to improve multi-turn goal-oriented conversations by balancing information gathering and commitment.

Contribution

It formulates conversation as an uncertainty-aware decision process and integrates language models with structured planning for better long-term decision making.

Findings

CUP improves success rates across multiple benchmarks.

It reduces the number of interaction turns needed.

Uncertainty-aware planning enhances early confident commitment.

Abstract

Goal-oriented conversational systems require making sequential decisions under uncertainty about the user's intent, where the algorithm must balance information acquisition and target commitment over multiple turns. Existing approaches address this challenge from different perspectives: structured methods enable multi-step planning but rely on predefined schemas, while LLM-based approaches support flexible interactions but lack long-horizon decision making, resulting in poor coordination between information acquisition and target commitment. To address this limitation, we formulate goal-oriented conversation as an uncertainty-aware sequential decision problem, where uncertainty serves as a guiding signal for multi-turn decision making. We propose a Conversation Uncertainty-aware Planning framework (CUP) that integrates language models with structured planning: a language model proposes feasible actions, and a planner evaluates their long-term impact on uncertainty reduction. Experiments on multiple conversational benchmarks show that CUP consistently improves success rates while requiring fewer interaction turns. Further analysis demonstrates that uncertainty-aware planning contributes to more efficient information acquisition and earlier confident commitment.