Thinking Clearly, Talking Fast: Concept-Guided Non-Autoregressive Generation for Open-Domain Dialogue Systems

TL;DR

This paper introduces a concept-guided non-autoregressive model for open-domain dialogue systems that improves response diversity, coherence, and inference speed by effectively managing multiple concepts during response generation.

Contribution

The paper presents a novel multi-concept planning module and a customized Insertion Transformer for non-autoregressive dialogue generation, enabling better concept management and faster responses.

Findings

Outperforms state-of-the-art baselines in automatic evaluations

Produces more diverse and coherent responses

Achieves substantially faster inference speed

Abstract

Human dialogue contains evolving concepts, and speakers naturally associate multiple concepts to compose a response. However, current dialogue models with the seq2seq framework lack the ability to effectively manage concept transitions and can hardly introduce multiple concepts to responses in a sequential decoding manner. To facilitate a controllable and coherent dialogue, in this work, we devise a concept-guided non-autoregressive model (CG-nAR) for open-domain dialogue generation. The proposed model comprises a multi-concept planning module that learns to identify multiple associated concepts from a concept graph and a customized Insertion Transformer that performs concept-guided non-autoregressive generation to complete a response. The experimental results on two public datasets show that CG-nAR can produce diverse and coherent responses, outperforming state-of-the-art baselines in both automatic and human evaluations with substantially faster inference speed.