Training Neural Response Selection for Task-Oriented Dialogue Systems

TL;DR

This paper introduces a two-stage training approach for response selection in task-oriented dialogue systems, leveraging large general-domain pretraining followed by domain-specific fine-tuning to improve performance in low-data scenarios.

Contribution

It presents a novel pretraining and fine-tuning method that enhances retrieval-based response selection for task-oriented dialogues with limited in-domain data.

Findings

Effective response selection across six diverse domains

Significant performance improvements over baseline models

Demonstrated adaptability to various application areas

Abstract

Despite their popularity in the chatbot literature, retrieval-based models have had modest impact on task-oriented dialogue systems, with the main obstacle to their application being the low-data regime of most task-oriented dialogue tasks. Inspired by the recent success of pretraining in language modelling, we propose an effective method for deploying response selection in task-oriented dialogue. To train response selection models for task-oriented dialogue tasks, we propose a novel method which: 1) pretrains the response selection model on large general-domain conversational corpora; and then 2) fine-tunes the pretrained model for the target dialogue domain, relying only on the small in-domain dataset to capture the nuances of the given dialogue domain. Our evaluation on six diverse application domains, ranging from e-commerce to banking, demonstrates the effectiveness of the proposed training method.