Adversarial Decomposition of Text Representation

TL;DR

This paper introduces an adversarial method for decomposing text representations into independent vectors for specific aspects, enabling controlled style modifications and improved paraphrase detection.

Contribution

It presents a novel adversarial decomposition technique that captures continuous style representations and enhances downstream NLP tasks.

Findings

Effective in converting social registers and language change

Learns continuous, linguistically realistic style representations

Outperforms autoencoder embeddings in paraphrase detection

Abstract

In this paper, we present a method for adversarial decomposition of text representation. This method can be used to decompose a representation of an input sentence into several independent vectors, each of them responsible for a specific aspect of the input sentence. We evaluate the proposed method on two case studies: the conversion between different social registers and diachronic language change. We show that the proposed method is capable of fine-grained controlled change of these aspects of the input sentence. It is also learning a continuous (rather than categorical) representation of the style of the sentence, which is more linguistically realistic. The model uses adversarial-motivational training and includes a special motivational loss, which acts opposite to the discriminator and encourages a better decomposition. Furthermore, we evaluate the obtained meaning embeddings on a downstream task of paraphrase detection and show that they significantly outperform the embeddings of a regular autoencoder.