DialogXpert: Driving Intelligent and Emotion-Aware Conversations through Online Value-Based Reinforcement Learning with LLM Priors

TL;DR

DialogXpert introduces an emotion-aware, goal-driven dialogue system that combines LLMs with reinforcement learning to enable efficient, empathetic, and strategic conversations across various tasks, achieving high success rates.

Contribution

It presents a novel framework integrating frozen LLMs with a compact Q-network for proactive, emotion-aware dialogue planning, improving over reactive models.

Findings

Achieves over 94% success rate in multi-turn conversations

Reduces conversation length to under 3 turns

Enhances negotiation outcomes with larger LLM priors

Abstract

Large-language-model (LLM) agents excel at reactive dialogue but struggle with proactive, goal-driven interactions due to myopic decoding and costly planning. We introduce DialogXpert, which leverages a frozen LLM to propose a small, high-quality set of candidate actions per turn and employs a compact Q-network over fixed BERT embeddings trained via temporal-difference learning to select optimal moves within this reduced space. By tracking the user's emotions, DialogXpert tailors each decision to advance the task while nurturing a genuine, empathetic connection. Across negotiation, emotional support, and tutoring benchmarks, DialogXpert drives conversations to under $3$ turns with success rates exceeding 94\% and, with a larger LLM prior, pushes success above 97\% while markedly improving negotiation outcomes. This framework delivers real-time, strategic, and emotionally intelligent dialogue planning at scale. Code available at https://github.com/declare-lab/dialogxpert/