Algorithms that get old : the case of generative deep neural networks

TL;DR

This paper explores how generative neural networks like VAEs can be adapted to produce diverse outputs that evolve over time, avoiding repetition and better mimicking human artistic progression.

Contribution

It introduces a numerical paradigm based on Radon-Sobolev distances to enhance generative models, ensuring non-repetitive, evolving outputs that cover the entire target distribution.

Findings

Proposes a Radon-Sobolev based method for generative diversity.

Ensures generated objects do not repeat and evolve over time.

Fills the entire target probability distribution with generated samples.

Abstract

Generative deep neural networks used in machine learning, like the Variational Auto-Encoders (VAE), and Generative Adversarial Networks (GANs) produce new objects each time when asked to do so with the constraint that the new objects remain similar to some list of examples given as input. However, this behavior is unlike that of human artists that change their style as time goes by and seldom return to the style of the initial creations. We investigate a situation where VAEs are used to sample from a probability measure described by some empirical dataset. Based on recent works on Radon-Sobolev statistical distances, we propose a numerical paradigm, to be used in conjunction with a generative algorithm, that satisfies the two following requirements: the objects created do not repeat and evolve to fill the entire target probability distribution.