# TextKD-GAN: Text Generation using KnowledgeDistillation and Generative Adversarial Networks

## TL;DR

This paper introduces TextKD-GAN, a novel approach combining knowledge distillation and GANs for improved text generation, leveraging autoencoders for continuous sentence representations to enhance quality and diversity.

## Contribution

It presents a new method that uses knowledge distillation with GANs and autoencoders to generate higher quality text, addressing challenges of discrete text data.

## Key findings

- Improved BLEU scores over traditional GAN-based methods
- Better Jensen-Shannon distance (JSD) measures achieved
- Effective use of autoencoders for continuous sentence representation

## Abstract

Text generation is of particular interest in many NLP applications such as machine translation, language modeling, and text summarization. Generative adversarial networks (GANs) achieved a remarkable success in high quality image generation in computer vision,and recently, GANs have gained lots of interest from the NLP community as well. However, achieving similar success in NLP would be more challenging due to the discrete nature of text. In this work, we introduce a method using knowledge distillation to effectively exploit GAN setup for text generation. We demonstrate how autoencoders (AEs) can be used for providing a continuous representation of sentences, which is a smooth representation that assign non-zero probabilities to more than one word. We distill this representation to train the generator to synthesize similar smooth representations. We perform a number of experiments to validate our idea using different datasets and show that our proposed approach yields better performance in terms of the BLEU score and Jensen-Shannon distance (JSD) measure compared to traditional GAN-based text generation approaches without pre-training.