TextSETTR: Few-Shot Text Style Extraction and Tunable Targeted Restyling

TL;DR

This paper introduces TextSETTR, a style transfer method that leverages unlabeled text and a pretrained model to perform targeted style adjustments across multiple style dimensions with minimal supervision.

Contribution

It presents a novel style transfer approach that uses unlabeled data and a pretrained T5 model to enable targeted, multi-dimensional style restyling without requiring labeled training data.

Findings

Competitive sentiment transfer performance with unlabeled training data.

Effective multi-style transfer across dialect, emotiveness, formality, politeness, and sentiment.

Single model handles diverse style dimensions with few exemplars at inference.

Abstract

We present a novel approach to the problem of text style transfer. Unlike previous approaches requiring style-labeled training data, our method makes use of readily-available unlabeled text by relying on the implicit connection in style between adjacent sentences, and uses labeled data only at inference time. We adapt T5 (Raffel et al., 2020), a strong pretrained text-to-text model, to extract a style vector from text and use it to condition the decoder to perform style transfer. As our label-free training results in a style vector space encoding many facets of style, we recast transfers as "targeted restyling" vector operations that adjust specific attributes of the input while preserving others. We demonstrate that training on unlabeled Amazon reviews data results in a model that is competitive on sentiment transfer, even compared to models trained fully on labeled data. Furthermore, applying our novel method to a diverse corpus of unlabeled web text results in a single model capable of transferring along multiple dimensions of style (dialect, emotiveness, formality, politeness, sentiment) despite no additional training and using only a handful of exemplars at inference time.