Exploiting Social Media Content for Self-Supervised Style Transfer

TL;DR

This paper introduces a novel self-supervised style transfer model that combines SSNMT and UNMT techniques to effectively utilize social media data for improved style transfer across multiple tasks.

Contribution

The paper presents the 3ST model, integrating SSNMT with UNMT methods, to better exploit supervisory signals in non-parallel social media data for style transfer.

Findings

3ST outperforms state-of-the-art models in automatic and human evaluations.

Balances fluency, content preservation, and attribute transfer more effectively.

Excels across civil rephrasing, formality, and polarity tasks.

Abstract

Recent research on style transfer takes inspiration from unsupervised neural machine translation (UNMT), learning from large amounts of non-parallel data by exploiting cycle consistency loss, back-translation, and denoising autoencoders. By contrast, the use of self-supervised NMT (SSNMT), which leverages (near) parallel instances hidden in non-parallel data more efficiently than UNMT, has not yet been explored for style transfer. In this paper we present a novel Self-Supervised Style Transfer (3ST) model, which augments SSNMT with UNMT methods in order to identify and efficiently exploit supervisory signals in non-parallel social media posts. We compare 3ST with state-of-the-art (SOTA) style transfer models across civil rephrasing, formality and polarity tasks. We show that 3ST is able to balance the three major objectives (fluency, content preservation, attribute transfer accuracy) the best, outperforming SOTA models on averaged performance across their tested tasks in automatic and human evaluation.