From Unsupervised Machine Translation To Adversarial Text Generation

TL;DR

This paper introduces B-GAN, a bilingual adversarial text generator that leverages unsupervised neural machine translation components to produce fluent, multilingual sentences using only monolingual data and multiple training losses.

Contribution

It presents a novel self-attention based bilingual adversarial text generator that combines auto-encoding, translation, and adversarial training for unsupervised multilingual text generation.

Findings

B-GAN generates more fluent sentences than monolingual baselines.

It effectively uses half the parameters of comparable models.

It can produce sentences in either language with shared encoder.

Abstract

We present a self-attention based bilingual adversarial text generator (B-GAN) which can learn to generate text from the encoder representation of an unsupervised neural machine translation system. B-GAN is able to generate a distributed latent space representation which can be paired with an attention based decoder to generate fluent sentences. When trained on an encoder shared between two languages and paired with the appropriate decoder, it can generate sentences in either language. B-GAN is trained using a combination of reconstruction loss for auto-encoder, a cross domain loss for translation and a GAN based adversarial loss for text generation. We demonstrate that B-GAN, trained on monolingual corpora only using multiple losses, generates more fluent sentences compared to monolingual baselines while effectively using half the number of parameters.