A Model-Agnostic Data Manipulation Method for Persona-based Dialogue Generation

TL;DR

This paper introduces a model-agnostic data manipulation technique that enhances persona-based dialogue generation by distilling and diversifying training data, leading to improved performance across different models.

Contribution

It presents a novel data manipulation approach that improves persona-based dialogue models by distilling and diversifying training data, applicable to any base model.

Findings

Our method outperforms baseline models in dialogue quality.

It improves training efficiency and model robustness.

Effective across multiple base models like Transformer and GPT2.

Abstract

Towards building intelligent dialogue agents, there has been a growing interest in introducing explicit personas in generation models. However, with limited persona-based dialogue data at hand, it may be difficult to train a dialogue generation model well. We point out that the data challenges of this generation task lie in two aspects: first, it is expensive to scale up current persona-based dialogue datasets; second, each data sample in this task is more complex to learn with than conventional dialogue data. To alleviate the above data issues, we propose a data manipulation method, which is model-agnostic to be packed with any persona-based dialogue generation model to improve its performance. The original training samples will first be distilled and thus expected to be fitted more easily. Next, we show various effective ways that can diversify such easier distilled data. A given base model will then be trained via the constructed data curricula, i.e. first on augmented distilled samples and then on original ones. Experiments illustrate the superiority of our method with two strong base dialogue models (Transformer encoder-decoder and GPT2).