Retrieval-Augmented Transformer-XL for Close-Domain Dialog Generation

TL;DR

This paper introduces a transformer-based dialog generation model enhanced with a retrieval mechanism that uses k-Nearest Neighbor search to improve response quality in goal-oriented conversations, evaluated on public and proprietary datasets.

Contribution

The paper presents a novel hybrid retrieval-augmented transformer model for multi-turn dialog generation, combining generative and retrieval-based approaches for improved performance.

Findings

Achieved higher BLEU scores than baselines on Taskmaster-1 and proprietary datasets.

Demonstrated effectiveness of retrieval augmentation in goal-oriented dialog tasks.

Improved response relevance and coherence in multi-turn conversations.

Abstract

Transformer-based models have demonstrated excellent capabilities of capturing patterns and structures in natural language generation and achieved state-of-the-art results in many tasks. In this paper we present a transformer-based model for multi-turn dialog response generation. Our solution is based on a hybrid approach which augments a transformer-based generative model with a novel retrieval mechanism, which leverages the memorized information in the training data via k-Nearest Neighbor search. Our system is evaluated on two datasets made by customer/assistant dialogs: the Taskmaster-1, released by Google and holding high quality, goal-oriented conversational data and a proprietary dataset collected from a real customer service call center. Both achieve better BLEU scores over strong baselines.