Generating Text through Adversarial Training using Skip-Thought Vectors

TL;DR

This paper introduces a novel text generation method combining Skip-Thought sentence embeddings with GANs, achieving improved performance in style reproduction and language generation tasks, validated through automated metrics and human judgment.

Contribution

It presents a new GAN-based architecture utilizing Skip-Thought embeddings for sentence-level text generation, outperforming existing models on multiple evaluation metrics.

Findings

Outperforms baseline models on BLEU-n, METEOR, and ROUGE scores

Effective in reproducing author writing styles at sentence level

Validated by human judgment scores in real-world tasks

Abstract

GANs have been shown to perform exceedingly well on tasks pertaining to image generation and style transfer. In the field of language modelling, word embeddings such as GLoVe and word2vec are state-of-the-art methods for applying neural network models on textual data. Attempts have been made to utilize GANs with word embeddings for text generation. This study presents an approach to text generation using Skip-Thought sentence embeddings with GANs based on gradient penalty functions and f-measures. The proposed architecture aims to reproduce writing style in the generated text by modelling the way of expression at a sentence level across all the works of an author. Extensive experiments were run in different embedding settings on a variety of tasks including conditional text generation and language generation. The model outperforms baseline text generation networks across several automated evaluation metrics like BLEU-n, METEOR and ROUGE. Further, wide applicability and effectiveness in real life tasks are demonstrated through human judgement scores.