Towards Robust Online Dialogue Response Generation

TL;DR

This paper addresses the inconsistency in multi-turn dialogue response generation by proposing hierarchical sampling, reinforcement learning, and re-ranking methods to improve chatbot robustness in real-world scenarios.

Contribution

It introduces a hierarchical sampling approach combined with reinforcement learning and re-ranking to reduce training-testing discrepancy and enhance dialogue coherence.

Findings

Improved robustness of chatbots in real-world settings.

Enhanced dialogue coherence through proposed methods.

Empirical validation shows significant performance gains.

Abstract

Although pre-trained sequence-to-sequence models have achieved great success in dialogue response generation, chatbots still suffer from generating inconsistent responses in real-world practice, especially in multi-turn settings. We argue that this can be caused by a discrepancy between training and real-world testing. At training time, chatbot generates the response with the golden context, while it has to generate based on the context consisting of both user utterances and the model predicted utterances during real-world testing. With the growth of the number of utterances, this discrepancy becomes more serious in the multi-turn settings. In this paper, we propose a hierarchical sampling-based method consisting of both utterance-level sampling and semi-utterance-level sampling, to alleviate the discrepancy, which implicitly increases the dialogue coherence. We further adopt reinforcement learning and re-ranking methods to explicitly optimize the dialogue coherence during training and inference, respectively. Empirical experiments show the effectiveness of the proposed methods for improving the robustness of chatbots in real practice.