On foundation of generative statistics with G-entropy: a gradient-based approach

TL;DR

This paper introduces a new G-entropy related to Fisher divergence to enhance gradient-based generative modeling, enabling better handling of missing data and model selection in high-dimensional data generation.

Contribution

It proposes a rigorous foundation for gradient-based generative statistics using G-entropy, expanding its applicability to model selection and handling missing data.

Findings

Introduced G-entropy related to Fisher divergence.

Developed a gradient-based algorithm for generative modeling.

Enabled model selection using the new framework.

Abstract

This paper explores the interplay between statistics and generative artificial intelligence. Generative statistics, an integral part of the latter, aims to construct models that can generate efficiently and meaningfully new data across the whole of the (usually high dimensional) sample space, e.g. a new photo. Within it, the gradient-based approach is a current favourite that exploits effectively, for the above purpose, the information contained in the observed sample, e.g. an old photo. However, often there are missing data in the observed sample, e.g., missing bits in the old photo. To handle this situation, we have proposed a gradient-based algorithm for generative modelling. More importantly, our paper underpins rigorously this powerful approach by introducing a new G-entropy that is related to the Fisher divergence. (The G-entropy is also of independent interest.) The underpinning has enabled the gradient-based approach to expand its scope. For example, it can now provide a tool for generative model selection. Possible future projects include discrete data and Bayesian variational inference.