Wavelets to the Rescue: Improving Sample Quality of Latent Variable Deep Generative Models

TL;DR

This paper introduces a wavelet space VAE that models images in wavelet coefficient space, leading to higher quality image generation with better detail preservation and competitive FID scores compared to GANs.

Contribution

The novel wavelet space VAE emphasizes high-frequency details by modeling wavelet coefficients, improving sample quality over traditional VAEs.

Findings

Achieves better FID scores than standard VAEs on natural image datasets.

Generates images with higher detail and less blurriness.

Retains disentangled and informative latent representations.

Abstract

Variational Autoencoders (VAE) are probabilistic deep generative models underpinned by elegant theory, stable training processes, and meaningful manifold representations. However, they produce blurry images due to a lack of explicit emphasis over high-frequency textural details of the images, and the difficulty to directly model the complex joint probability distribution over the high-dimensional image space. In this work, we approach these two challenges with a novel wavelet space VAE that uses the decoder to model the images in the wavelet coefficient space. This enables the VAE to emphasize over high-frequency components within an image obtained via wavelet decomposition. Additionally, by decomposing the complex function of generating high-dimensional images into inverse wavelet transformation and generation of wavelet coefficients, the latter becomes simpler to model by the VAE. We empirically validate that deep generative models operating in the wavelet space can generate images of higher quality than the image (RGB) space counterparts. Quantitatively, on benchmark natural image datasets, we achieve consistently better FID scores than VAE based architectures and competitive FID scores with a variety of GAN models for the same architectural and experimental setup. Furthermore, the proposed wavelet-based generative model retains desirable attributes like disentangled and informative latent representation without losing the quality in the generated samples.