StyleDGPT: Stylized Response Generation with Pre-trained Language Models

TL;DR

StyleDGPT leverages pre-trained language models with a novel KL loss and style classifier to generate responses that adhere to specific styles, improving style consistency and coherence in dialogue systems.

Contribution

This work introduces a new fine-tuning approach with KL loss and style classifier for stylized response generation using pre-trained models.

Findings

Significant improvement in style consistency over state-of-the-art methods.

Enhanced contextual coherence in stylized responses.

Effective style control at word and sentence levels.

Abstract

Generating responses following a desired style has great potentials to extend applications of open-domain dialogue systems, yet is refrained by lacking of parallel data for training. In this work, we explore the challenging task with pre-trained language models that have brought breakthrough to various natural language tasks. To this end, we introduce a KL loss and a style classifier to the fine-tuning step in order to steer response generation towards the target style in both a word-level and a sentence-level. Comprehensive empirical studies with two public datasets indicate that our model can significantly outperform state-of-the-art methods in terms of both style consistency and contextual coherence.