An Overview on Controllable Text Generation via Variational Auto-Encoders

TL;DR

This paper reviews the use of variational auto-encoders for controllable text generation, discussing existing methods, challenges, applications, datasets, and future research directions in this field.

Contribution

It provides a comprehensive overview of controllable text generation using VAEs, highlighting current methodologies, problems, and potential future research avenues.

Findings

Summarizes existing VAE-based controllable text generation schemes.

Identifies key challenges and problems in current approaches.

Discusses datasets, metrics, and future research directions.

Abstract

Recent advances in neural-based generative modeling have reignited the hopes of having computer systems capable of conversing with humans and able to understand natural language. The employment of deep neural architectures has been largely explored in a multitude of context and tasks to fulfill various user needs. On one hand, producing textual content that meets specific requirements is of priority for a model to seamlessly conduct conversations with different groups of people. On the other hand, latent variable models (LVM) such as variational auto-encoders (VAEs) as one of the most popular genres of generative models are designed to characterize the distributional pattern of textual data. Thus they are inherently capable of learning the integral textual features that are worth exploring for controllable pursuits. \noindent This overview gives an introduction to existing generation schemes, problems associated with text variational auto-encoders, and a review of several applications about the controllable generation that are instantiations of these general formulations,\footnote{A detailed paper list is available at \url{https://github.com/ImKeTT/CTG-latentAEs}} as well as related datasets, metrics and discussions for future researches. Hopefully, this overview will provide an overview of living questions, popular methodologies and raw thoughts for controllable language generation under the scope of variational auto-encoder.