Symmetries and control in generative neural nets

TL;DR

This paper explores how incorporating learned symmetry statistics into generative neural networks enhances their ability to generate, control, and modify observations, demonstrated on distorted MNIST and CIFAR10 datasets.

Contribution

It introduces a method to integrate symmetry statistics as gauge groups into auto-encoder-based generative models, improving their generative and control capabilities.

Findings

Enhanced generative quality with symmetry statistics

Improved control over object attributes in generated images

Effective on distorted MNIST and CIFAR10 datasets

Abstract

We study generative nets which can control and modify observations, after being trained on real-life datasets. In order to zoom-in on an object, some spatial, color and other attributes are learned by classifiers in specialized attention nets. In field-theoretical terms, these learned symmetry statistics form the gauge group of the data set. Plugging them in the generative layers of auto-classifiers-encoders (ACE) appears to be the most direct way to simultaneously: i) generate new observations with arbitrary attributes, from a given class, ii) describe the low-dimensional manifold encoding the "essence" of the data, after superfluous attributes are factored out, and iii) organically control, i.e., move or modify objects within given observations. We demonstrate the sharp improvement of the generative qualities of shallow ACE, with added spatial and color symmetry statistics, on the distorted MNIST and CIFAR10 datasets.