Unsupervised Neural Stylistic Text Generation using Transfer learning and Adapters

TL;DR

This paper introduces a transfer learning approach using adapters that updates only a tiny fraction of model parameters to generate stylistic responses, improving style consistency without extensive data annotation.

Contribution

It presents a novel transfer learning framework with adapters that efficiently learn style-specific attributes for text generation, reducing training effort and data requirements.

Findings

200% improvement in style generation quality over baseline models

Maintains content relevance comparable to standard encoder-decoder models

Efficiently learns style with only 0.3% of model parameters updated

Abstract

Research has shown that personality is a key driver to improve engagement and user experience in conversational systems. Conversational agents should also maintain a consistent persona to have an engaging conversation with a user. However, text generation datasets are often crowd sourced and thereby have an averaging effect where the style of the generation model is an average style of all the crowd workers that have contributed to the dataset. While one can collect persona-specific datasets for each task, it would be an expensive and time consuming annotation effort. In this work, we propose a novel transfer learning framework which updates only $0.3\%$ of model parameters to learn style specific attributes for response generation. For the purpose of this study, we tackle the problem of stylistic story ending generation using the ROC stories Corpus. We learn style specific attributes from the PERSONALITY-CAPTIONS dataset. Through extensive experiments and evaluation metrics we show that our novel training procedure can improve the style generation by 200 over Encoder-Decoder baselines while maintaining on-par content relevance metrics with