Preventing Oversmoothing in VAE via Generalized Variance Parameterization

TL;DR

This paper introduces VAE extensions with generalized variance parameterization and adaptive regularization to prevent oversmoothing and posterior collapse, leading to improved image quality on benchmark datasets.

Contribution

It proposes novel VAE extensions that adaptively regularize decoder smoothness using maximum likelihood estimation, addressing oversmoothing issues.

Findings

Improved FID scores on MNIST and CelebA datasets.

Effective prevention of oversmoothing and posterior collapse.

Enhanced latent space informativeness.

Abstract

Variational autoencoders (VAEs) often suffer from posterior collapse, which is a phenomenon in which the learned latent space becomes uninformative. This is often related to the hyperparameter resembling the data variance. It can be shown that an inappropriate choice of this hyperparameter causes the oversmoothness in the linearly approximated case and can be empirically verified for the general cases. Moreover, determining such appropriate choice becomes infeasible if the data variance is non-uniform or conditional. Therefore, we propose VAE extensions with generalized parameterizations of the data variance and incorporate maximum likelihood estimation into the objective function to adaptively regularize the decoder smoothness. The images generated from proposed VAE extensions show improved Fr\'echet inception distance (FID) on MNIST and CelebA datasets.