InstructTODS: Large Language Models for End-to-End Task-Oriented Dialogue Systems

TL;DR

InstructTODS introduces a zero-shot, end-to-end task-oriented dialogue system leveraging large language models, achieving comparable performance to fine-tuned models and outperforming existing systems in human evaluations without requiring task-specific training.

Contribution

This work presents InstructTODS, a novel framework that uses LLMs for zero-shot end-to-end TODS, enabling domain adaptation without fine-tuning and demonstrating superior human evaluation results.

Findings

Achieves comparable performance to fine-tuned TODS

Outperforms state-of-the-art TODS in human evaluations

Effectively handles dialogue state tracking, intent classification, and response generation

Abstract

Large language models (LLMs) have been used for diverse tasks in natural language processing (NLP), yet remain under-explored for task-oriented dialogue systems (TODS), especially for end-to-end TODS. We present InstructTODS, a novel off-the-shelf framework for zero-shot end-to-end task-oriented dialogue systems that can adapt to diverse domains without fine-tuning. By leveraging LLMs, InstructTODS generates a proxy belief state that seamlessly translates user intentions into dynamic queries for efficient interaction with any KB. Our extensive experiments demonstrate that InstructTODS achieves comparable performance to fully fine-tuned TODS in guiding dialogues to successful completion without prior knowledge or task-specific data. Furthermore, a rigorous human evaluation of end-to-end TODS shows that InstructTODS produces dialogue responses that notably outperform both the gold responses and the state-of-the-art TODS in terms of helpfulness, informativeness, and humanness. Moreover, the effectiveness of LLMs in TODS is further supported by our comprehensive evaluations on TODS subtasks: dialogue state tracking, intent classification, and response generation. Code and implementations could be found here https://github.com/WillyHC22/InstructTODS/