Measuring global properties of neural generative model outputs via generating mathematical objects

TL;DR

This paper evaluates deep generative models trained on complete datasets of reflexive polytopes, demonstrating their ability to learn complex geometric properties and distinguish between memorization and genuine understanding.

Contribution

It introduces a method to assess global geometric properties learned by generative models using complete datasets of reflexive polytopes and compares different data representations.

Findings

Models can generate geometric objects with non-trivial properties

Models learn underlying object properties beyond memorization

Different data representations affect learning ease

Abstract

We train deep generative models on datasets of reflexive polytopes. This enables us to compare how well the models have picked up on various global properties of generated samples. Our datasets are complete in the sense that every single example, up to changes of coordinate, is included in the dataset. Using this property we also perform tests checking to what extent the models are merely memorizing the data. We also train models on the same dataset represented in two different ways, enabling us to measure which form is easiest to learn from. We use these experiments to show that deep generative models can learn to generate geometric objects with non-trivial global properties, and that the models learn some underlying properties of the objects rather than simply memorizing the data.