Generating new pictures in complex datasets with a simple neural network

TL;DR

This paper presents a simple variational auto-encoder with minimal latent dimensions that can generate realistic perturbations of images in complex datasets like CIFAR-10, despite not reconstructing all images perfectly.

Contribution

The authors introduce a VAE with only two latent dimensions per class, enabling effective image perturbations in complex datasets, which is a novel approach.

Findings

Effective image perturbation in CIFAR-10

Requires classifier to identify well reconstructed images

Limited reconstruction quality for some training images

Abstract

We introduce a version of a variational auto-encoder (VAE), which can generate good perturbations of images, when trained on a complex dataset (in our experiments, CIFAR-10). The net is using only two latent generative dimensions per class, with uni-modal probability density. The price one has to pay for good generation is that not all training images are well reconstructed. An additional classifier is required to determine which training image is well reconstructed and generally the weights of training images. Only training images which are well reconstructed, can be perturbed. For good perturbations, we use the tentative empirical drifts of well reconstructed images. The construct is not predictive in the usual statistical sense.