LED: Latent Variable-based Estimation of Density

TL;DR

LED is a novel generative model that combines efficient sampling with accurate density estimation by leveraging a latent variable approach and an adversarial training objective, bridging the gap between high-quality sample generation and exact density computation.

Contribution

We introduce LED, a new model that enables both efficient sampling and exact density estimation, integrating flow-based generators with adversarial training.

Findings

LED provides accurate density estimates on various datasets.

The model maintains high-quality sample generation.

It offers insights into relationships among popular generative models.

Abstract

Modern generative models are roughly divided into two main categories: (1) models that can produce high-quality random samples, but cannot estimate the exact density of new data points and (2) those that provide exact density estimation, at the expense of sample quality and compactness of the latent space. In this work we propose LED, a new generative model closely related to GANs, that allows not only efficient sampling but also efficient density estimation. By maximizing log-likelihood on the output of the discriminator, we arrive at an alternative adversarial optimization objective that encourages generated data diversity. This formulation provides insights into the relationships between several popular generative models. Additionally, we construct a flow-based generator that can compute exact probabilities for generated samples, while allowing low-dimensional latent variables as input. Our experimental results, on various datasets, show that our density estimator produces accurate estimates, while retaining good quality in the generated samples.