SALSA-TEXT : self attentive latent space based adversarial text generation

TL;DR

This paper introduces SALSA-TEXT, a novel self-attention-based architecture that enhances adversarial latent space models for text generation, outperforming existing methods like AAE and ARAE on benchmark datasets.

Contribution

The paper proposes a new self-attention architecture for adversarial latent code-based text generation, improving performance over prior models like AAE and ARAE.

Findings

Self-attention models outperform state-of-the-art in adversarial text generation.

The proposed models achieve better objective and subjective evaluation scores.

Experimental results on sentence compression dataset validate the effectiveness of SALSA-TEXT.

Abstract

Inspired by the success of self attention mechanism and Transformer architecture in sequence transduction and image generation applications, we propose novel self attention-based architectures to improve the performance of adversarial latent code- based schemes in text generation. Adversarial latent code-based text generation has recently gained a lot of attention due to their promising results. In this paper, we take a step to fortify the architectures used in these setups, specifically AAE and ARAE. We benchmark two latent code-based methods (AAE and ARAE) designed based on adversarial setups. In our experiments, the Google sentence compression dataset is utilized to compare our method with these methods using various objective and subjective measures. The experiments demonstrate the proposed (self) attention-based models outperform the state-of-the-art in adversarial code-based text generation.