ControlVAE: Controllable Variational Autoencoder
Huajie Shao, Shuochao Yao, Dachun Sun, Aston Zhang, Shengzhong Liu,, Dongxin Liu, Jun Wang, Tarek Abdelzaher

TL;DR
ControlVAE introduces a control-theoretic approach to variational autoencoders, using a PI controller to automatically tune hyperparameters, resulting in improved disentangling, reconstruction, and diversity in generative tasks.
Contribution
It presents a novel framework combining a PI controller with VAE to automatically optimize training hyperparameters, addressing KL vanishing and enhancing generative quality.
Findings
Achieves better disentangling and reconstruction quality.
Prevents KL vanishing in language modeling.
Improves diversity of generated text and image reconstructions.
Abstract
Variational Autoencoders (VAE) and their variants have been widely used in a variety of applications, such as dialog generation, image generation and disentangled representation learning. However, the existing VAE models have some limitations in different applications. For example, a VAE easily suffers from KL vanishing in language modeling and low reconstruction quality for disentangling. To address these issues, we propose a novel controllable variational autoencoder framework, ControlVAE, that combines a controller, inspired by automatic control theory, with the basic VAE to improve the performance of resulting generative models. Specifically, we design a new non-linear PI controller, a variant of the proportional-integral-derivative (PID) control, to automatically tune the hyperparameter (weight) added in the VAE objective using the output KL-divergence as feedback during model…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Music and Audio Processing · Topic Modeling
MethodsSolana Customer Service Number +1-833-534-1729 · USD Coin Customer Service Number +1-833-534-1729
