A Simple Generative Network

TL;DR

This paper introduces a simple generative neural network architecture that, despite its simplicity, produces competitive results compared to more complex models like GANs and VAEs, challenging the notion that complexity is necessary for high-quality generative modeling.

Contribution

The paper presents a straightforward, single feed-forward neural network model using KL divergence, demonstrating competitive generative performance against complex architectures.

Findings

The simple model generates visually convincing samples.

Quantitative results are comparable to state-of-the-art methods.

The approach challenges the need for complex architectures in generative modeling.

Abstract

Generative neural networks are able to mimic intricate probability distributions such as those of handwritten text, natural images, etc. Since their inception several models were proposed. The most successful of these were based on adversarial (GAN), auto-encoding (VAE) and maximum mean discrepancy (MMD) relatively complex architectures and schemes. Surprisingly, a very simple architecture (a single feed-forward neural network) in conjunction with an obvious optimization goal (Kullback_Leibler divergence) was apparently overlooked. This paper demonstrates that such a model (denoted SGN for its simplicity) is able to generate samples visually and quantitatively competitive as compared with the fore-mentioned state of the art methods.