Transforming Delete, Retrieve, Generate Approach for Controlled Text Style Transfer

TL;DR

This paper introduces the Generative Style Transformer (GST), a novel method for controlled text style transfer that leverages large pre-trained models and a delete-retrieve-generate framework, outperforming existing methods on multiple datasets.

Contribution

The paper presents GST, a new style transfer model that does not require parallel corpora and introduces a Transformer-based deletion method within a delete-retrieve-generate framework.

Findings

GST outperforms state-of-the-art systems on 5 datasets.

The proposed GLEU metric correlates better with human judgments.

A novel Transformer-based deletion method improves style attribute removal.

Abstract

Text style transfer is the task of transferring the style of text having certain stylistic attributes, while preserving non-stylistic or content information. In this work we introduce the Generative Style Transformer (GST) - a new approach to rewriting sentences to a target style in the absence of parallel style corpora. GST leverages the power of both, large unsupervised pre-trained language models as well as the Transformer. GST is a part of a larger `Delete Retrieve Generate' framework, in which we also propose a novel method of deleting style attributes from the source sentence by exploiting the inner workings of the Transformer. Our models outperform state-of-art systems across 5 datasets on sentiment, gender and political slant transfer. We also propose the use of the GLEU metric as an automatic metric of evaluation of style transfer, which we found to compare better with human ratings than the predominantly used BLEU score.