Deploying a Retrieval based Response Model for Task Oriented Dialogues

TL;DR

This paper presents a scalable, adaptable retrieval-based response model for task-oriented dialogues, combining semi-automatic template creation, neural ranking architecture, and a two-stage training process, validated through offline and live customer interactions.

Contribution

It introduces a novel three-step approach including template generation, neural ranking with constraints, and a two-stage training strategy for task-oriented dialogue systems.

Findings

Effective response ranking in live customer interactions

High coverage template set created without annotations

Successful deployment with human-in-the-loop

Abstract

Task-oriented dialogue systems in industry settings need to have high conversational capability, be easily adaptable to changing situations and conform to business constraints. This paper describes a 3-step procedure to develop a conversational model that satisfies these criteria and can efficiently scale to rank a large set of response candidates. First, we provide a simple algorithm to semi-automatically create a high-coverage template set from historic conversations without any annotation. Second, we propose a neural architecture that encodes the dialogue context and applicable business constraints as profile features for ranking the next turn. Third, we describe a two-stage learning strategy with self-supervised training, followed by supervised fine-tuning on limited data collected through a human-in-the-loop platform. Finally, we describe offline experiments and present results of deploying our model with human-in-the-loop to converse with live customers online.