Tailored Conversations beyond LLMs: A RL-Based Dialogue Manager

TL;DR

This paper introduces a novel RL-based dialogue manager integrated with LLMs, designed for goal-oriented, personalized conversations, demonstrating improved performance over existing LLM-based systems in motivational interviewing scenarios.

Contribution

It presents a hierarchical reinforcement learning framework combined with meta-learning to enhance adaptability, efficiency, and personalization in open-ended dialogue systems beyond traditional LLM approaches.

Findings

Outperforms state-of-the-art LLM baseline in reward metrics

Enables fluid transition between dialogue phases

Personalizes responses to diverse user profiles

Abstract

In this work, we propose a novel framework that integrates large language models (LLMs) with an RL-based dialogue manager for open-ended dialogue with a specific goal. By leveraging hierarchical reinforcement learning to model the structured phases of dialogue and employ meta-learning to enhance adaptability across diverse user profiles, our approach enhances adaptability and efficiency, enabling the system to learn from limited data, transition fluidly between dialogue phases, and personalize responses to heterogeneous patient needs. We apply our framework to Motivational Interviews, aiming to foster behavior change, and demonstrate that the proposed dialogue manager outperforms a state-of-the-art LLM baseline in terms of reward, showing a potential benefit of conditioning LLMs to create open-ended dialogue systems with specific goals.