Promoting Open-domain Dialogue Generation through Learning Pattern Information between Contexts and Responses

TL;DR

This paper enhances open-domain dialogue generation by learning implicit pattern information between contexts and responses, using an improved scheduled sampling method and a response-aware mechanism with GPT-2, leading to more diverse and human-like replies.

Contribution

It introduces a novel response-aware mechanism and an improved scheduled sampling method to improve response diversity and contextual relevance in open-domain dialogue models.

Findings

Outperforms baselines on Persona-Chat and DailyDialog datasets

Produces more diverse and human-like responses

Improves automatic and manual evaluation metrics

Abstract

Recently, utilizing deep neural networks to build the opendomain dialogue models has become a hot topic. However, the responses generated by these models suffer from many problems such as responses not being contextualized and tend to generate generic responses that lack information content, damaging the user's experience seriously. Therefore, many studies try introducing more information into the dialogue models to make the generated responses more vivid and informative. Unlike them, this paper improves the quality of generated responses by learning the implicit pattern information between contexts and responses in the training samples. In this paper, we first build an open-domain dialogue model based on the pre-trained language model (i.e., GPT-2). And then, an improved scheduled sampling method is proposed for pre-trained models, by which the responses can be used to guide the response generation in the training phase while avoiding the exposure bias problem. More importantly, we design a response-aware mechanism for mining the implicit pattern information between contexts and responses so that the generated replies are more diverse and approximate to human replies. Finally, we evaluate the proposed model (RAD) on the Persona-Chat and DailyDialog datasets; and the experimental results show that our model outperforms the baselines on most automatic and manual metrics.