Unsupervised Extraction of Dialogue Policies from Conversations

TL;DR

This paper introduces a novel graph-based method leveraging Large Language Models to automatically extract and represent dialogue policies from conversational datasets, enhancing control and interpretability for dialogue system development.

Contribution

It presents a new approach combining LLMs and graph traversal to extract interpretable dialogue flows, improving over prompt-based methods.

Findings

Graph traversal yields better dialogue flow representations than LLM prompts.

The method provides greater control and interpretability for dialogue policy design.

Effective extraction of dialogue policies from large conversational datasets.

Abstract

Dialogue policies play a crucial role in developing task-oriented dialogue systems, yet their development and maintenance are challenging and typically require substantial effort from experts in dialogue modeling. While in many situations, large amounts of conversational data are available for the task at hand, people lack an effective solution able to extract dialogue policies from this data. In this paper, we address this gap by first illustrating how Large Language Models (LLMs) can be instrumental in extracting dialogue policies from datasets, through the conversion of conversations into a unified intermediate representation consisting of canonical forms. We then propose a novel method for generating dialogue policies utilizing a controllable and interpretable graph-based methodology. By combining canonical forms across conversations into a flow network, we find that running graph traversal algorithms helps in extracting dialogue flows. These flows are a better representation of the underlying interactions than flows extracted by prompting LLMs. Our technique focuses on giving conversation designers greater control, offering a productivity tool to improve the process of developing dialogue policies.