Diversity in deep generative models and generative AI

TL;DR

This paper introduces a kernel-based measure quantization method to enhance diversity in deep generative models, allowing for more varied object generation beyond traditional sampling methods.

Contribution

The paper proposes a novel kernel-based quantization technique that improves diversity in generative AI by approximating target distributions and avoiding repeated samples.

Findings

Enhanced diversity in generated objects demonstrated on benchmarks

Method effectively reduces sample repetition

Improved coverage of target distributions

Abstract

The decoder-based machine learning generative algorithms such as Generative Adversarial Networks (GAN), Variational Auto-Encoders (VAE), Transformers show impressive results when constructing objects similar to those in a training ensemble. However, the generation of new objects builds mainly on the understanding of the hidden structure of the training dataset followed by a sampling from a multi-dimensional normal variable. In particular each sample is independent from the others and can repeatedly propose same type of objects. To cure this drawback we introduce a kernel-based measure quantization method that can produce new objects from a given target measure by approximating it as a whole and even staying away from elements already drawn from that distribution. This ensures a better diversity of the produced objects. The method is tested on classic machine learning benchmarks.