Dialog Flow Induction for Constrainable LLM-Based Chatbots

TL;DR

This paper presents an unsupervised method to automatically generate domain-specific dialog flows that constrain large language model chatbots, improving their accuracy and relevance across various specialized fields.

Contribution

It introduces two variants of dialog flow induction based on in-domain data, reducing manual effort and enhancing domain coverage for LLM chatbots.

Findings

Higher domain coverage with data-guided flows

Better performance over manual dialog flow creation

Effective in multiple dialog domains

Abstract

LLM-driven dialog systems are used in a diverse set of applications, ranging from healthcare to customer service. However, given their generalization capability, it is difficult to ensure that these chatbots stay within the boundaries of the specialized domains, potentially resulting in inaccurate information and irrelevant responses. This paper introduces an unsupervised approach for automatically inducing domain-specific dialog flows that can be used to constrain LLM-based chatbots. We introduce two variants of dialog flow based on the availability of in-domain conversation instances. Through human and automatic evaluation over various dialog domains, we demonstrate that our high-quality data-guided dialog flows achieve better domain coverage, thereby overcoming the need for extensive manual crafting of such flows.