Maximum Entropy Generators for Energy-Based Models

TL;DR

This paper introduces a novel approach for energy-based models that learns an energy function and a generator network simultaneously, enabling efficient sampling and high-quality image generation without mode collapse.

Contribution

It proposes a method combining maximum entropy principles with neural generators and mutual information estimators to improve energy-based model training.

Findings

Generates sharp images with competitive Inception and FID scores.

Avoids mode collapse common in GANs.

Performs well in anomaly detection tasks.

Abstract

Maximum likelihood estimation of energy-based models is a challenging problem due to the intractability of the log-likelihood gradient. In this work, we propose learning both the energy function and an amortized approximate sampling mechanism using a neural generator network, which provides an efficient approximation of the log-likelihood gradient. The resulting objective requires maximizing entropy of the generated samples, which we perform using recently proposed nonparametric mutual information estimators. Finally, to stabilize the resulting adversarial game, we use a zero-centered gradient penalty derived as a necessary condition from the score matching literature. The proposed technique can generate sharp images with Inception and FID scores competitive with recent GAN techniques, does not suffer from mode collapse, and is competitive with state-of-the-art anomaly detection techniques.