Enhancing Large Language Model Induced Task-Oriented Dialogue Systems Through Look-Forward Motivated Goals

TL;DR

This paper introduces ProToD, a goal-driven approach for task-oriented dialogue systems that anticipates future actions and uses goal-oriented rewards, improving efficiency and user satisfaction with less data.

Contribution

The paper proposes ProToD, a novel proactive, goal-oriented framework for LLM-induced task-oriented dialogues, and a new evaluation method based on goal-driven simulations.

Findings

Achieves superior performance with only 10% of the data compared to previous models.

Enhances user satisfaction and dialogue efficiency.

Introduces a goal-driven evaluation method for ToD systems.

Abstract

Recently, the development of large language models (LLMs) has been significantly enhanced the question answering and dialogue generation, and makes them become increasingly popular in current practical scenarios. While unlike the general dialogue system which emphasizes the semantic performance, the task-oriented dialogue (ToD) systems aim to achieve the dialogue goal efficiently and successfully in multiple turns. Unfortunately, existing LLM-induced ToD systems lack the direct reward toward the final goal and do not take account of the dialogue proactivity that can strengthen the dialogue efficiency. To fill these gaps, we introduce the ProToD (Proactively Goal-Driven LLM-Induced ToD) approach, which anticipates the future dialogue actions and incorporates the goal-oriented reward signal to enhance ToD systems. Additionally, we present a novel evaluation method that assesses ToD systems based on goal-driven dialogue simulations. This method allows us to gauge user satisfaction, system efficiency and successful rate while overcoming the limitations of current Information and Success metrics. Empirical experiments conducted on the MultiWoZ 2.1 dataset demonstrate that our model can achieve superior performance using only 10% of the data compared to previous end-to-end fully supervised models. This improvement is accompanied by enhanced user satisfaction and efficiency.