Preventing posterior collapse in variational autoencoders for text generation via decoder regularization

TL;DR

This paper introduces a new regularization technique using fraternal dropout to prevent posterior collapse in variational autoencoders for text generation, leading to improved model performance.

Contribution

The paper proposes a novel fraternal dropout-based regularization method specifically designed to address posterior collapse in VAEs for text generation.

Findings

Improved metrics across multiple configurations

Effective prevention of posterior collapse

Enhanced text generation quality

Abstract

Variational autoencoders trained to minimize the reconstruction error are sensitive to the posterior collapse problem, that is the proposal posterior distribution is always equal to the prior. We propose a novel regularization method based on fraternal dropout to prevent posterior collapse. We evaluate our approach using several metrics and observe improvements in all the tested configurations.