Towards Robust Personalized Dialogue Generation via Order-Insensitive Representation Regularization

TL;DR

This paper addresses the order sensitivity issue in personalized dialogue generation by proposing a model-agnostic framework called ORIG, which enhances response consistency across different persona sentence orders, improving robustness of dialogue models.

Contribution

The paper introduces ORIG, a novel framework that reduces order sensitivity in dialogue models, leading to more consistent responses regardless of persona sentence order.

Findings

Significant performance fluctuations due to order sensitivity (29.4% on GPT2, 83.2% on BART)

ORIG effectively mitigates order sensitivity, improving response consistency

Experiments on Persona-Chat validate the superiority of ORIG with GPT2 and BART.

Abstract

Generating persona consistent dialogue response is important for developing an intelligent conversational agent. Recent works typically fine-tune large-scale pre-trained models on this task by concatenating persona texts and dialogue history as a single input sequence to generate the target response. While simple and effective, our analysis shows that this popular practice is seriously affected by order sensitivity where different input orders of persona sentences significantly impact the quality and consistency of generated response, resulting in severe performance fluctuations (i.e., 29.4% on GPT2 and 83.2% on BART). To mitigate the order sensitivity problem, we propose a model-agnostic framework, ORder Insensitive Generation (ORIG), which enables dialogue models to learn robust representation under different persona orders and improve the consistency of response generation. Experiments on the Persona-Chat dataset justify the effectiveness and superiority of our method with two dominant pre-trained models (GPT2 and BART).