Contextual Text Style Transfer

TL;DR

This paper introduces Contextual Text Style Transfer, a task that considers surrounding context for style transfer, and proposes the CAST model to improve semantic preservation and contextual consistency with limited data.

Contribution

The paper presents a novel task of contextual style transfer and a new model, CAST, that effectively incorporates context and semi-supervised learning for improved style transfer.

Findings

CAST outperforms state-of-the-art methods in style accuracy.

The model maintains better content and contextual consistency.

New benchmarks Enron-Context and Reddit-Context are introduced.

Abstract

We introduce a new task, Contextual Text Style Transfer - translating a sentence into a desired style with its surrounding context taken into account. This brings two key challenges to existing style transfer approaches: ($i$) how to preserve the semantic meaning of target sentence and its consistency with surrounding context during transfer; ($ii$) how to train a robust model with limited labeled data accompanied with context. To realize high-quality style transfer with natural context preservation, we propose a Context-Aware Style Transfer (CAST) model, which uses two separate encoders for each input sentence and its surrounding context. A classifier is further trained to ensure contextual consistency of the generated sentence. To compensate for the lack of parallel data, additional self-reconstruction and back-translation losses are introduced to leverage non-parallel data in a semi-supervised fashion. Two new benchmarks, Enron-Context and Reddit-Context, are introduced for formality and offensiveness style transfer. Experimental results on these datasets demonstrate the effectiveness of the proposed CAST model over state-of-the-art methods across style accuracy, content preservation and contextual consistency metrics.