Adversarial Likelihood Estimation With One-Way Flows

TL;DR

This paper introduces a novel one-way flow network for explicit density estimation in generative models, enabling unbiased likelihood estimation and improved mode coverage, with faster convergence and high-quality sample generation.

Contribution

It proposes a new one-way flow network architecture for explicit density estimation, allowing unbiased likelihood computation and better mode coverage in generative modeling.

Findings

Faster convergence compared to traditional GANs

Produces high-quality, smooth latent representations

Avoids overfitting on standard datasets

Abstract

Generative Adversarial Networks (GANs) can produce high-quality samples, but do not provide an estimate of the probability density around the samples. However, it has been noted that maximizing the log-likelihood within an energy-based setting can lead to an adversarial framework where the discriminator provides unnormalized density (often called energy). We further develop this perspective, incorporate importance sampling, and show that 1) Wasserstein GAN performs a biased estimate of the partition function, and we propose instead to use an unbiased estimator; and 2) when optimizing for likelihood, one must maximize generator entropy. This is hypothesized to provide a better mode coverage. Different from previous works, we explicitly compute the density of the generated samples. This is the key enabler to designing an unbiased estimator of the partition function and computation of the generator entropy term. The generator density is obtained via a new type of flow network, called one-way flow network, that is less constrained in terms of architecture, as it does not require a tractable inverse function. Our experimental results show that our method converges faster, produces comparable sample quality to GANs with similar architecture, successfully avoids over-fitting to commonly used datasets and produces smooth low-dimensional latent representations of the training data.