A Hybrid Convolutional Variational Autoencoder for Text Generation

TL;DR

This paper introduces a hybrid convolutional variational autoencoder architecture for text generation that improves training stability, speed, and handling of long sequences compared to traditional RNN-based VAEs.

Contribution

The paper proposes a novel hybrid architecture combining convolutional and recurrent components, enhancing VAE training and performance for text generation.

Findings

Faster runtime and convergence compared to traditional models

Improved handling of long sequences in text generation

Reduced training difficulties for VAE models on text data

Abstract

In this paper we explore the effect of architectural choices on learning a Variational Autoencoder (VAE) for text generation. In contrast to the previously introduced VAE model for text where both the encoder and decoder are RNNs, we propose a novel hybrid architecture that blends fully feed-forward convolutional and deconvolutional components with a recurrent language model. Our architecture exhibits several attractive properties such as faster run time and convergence, ability to better handle long sequences and, more importantly, it helps to avoid some of the major difficulties posed by training VAE models on textual data.