An information theoretic limit to data amplification

TL;DR

This paper establishes an information theoretic limit on data amplification using generative models like GANs, showing that increasing data sample size does not inherently improve variable resolution but can enhance statistical significance.

Contribution

It introduces a mathematical bound on data amplification, demonstrating that information content remains unchanged despite increased sample size, and clarifies conditions for this bound.

Findings

A gain greater than one is possible without increasing information content.

The resolution of variables is not improved by data amplification.

Sample size increase can improve statistical significance without extra information.

Abstract

In recent years generative artificial intelligence has been used to create data to support science analysis. For example, Generative Adversarial Networks (GANs) have been trained using Monte Carlo simulated input and then used to generate data for the same problem. This has the advantage that a GAN creates data in a significantly reduced computing time. N training events for a GAN can result in GN generated events with the gain factor, G, being more than one. This appears to violate the principle that one cannot get information for free. This is not the only way to amplify data so this process will be referred to as data amplification which is studied using information theoretic concepts. It is shown that a gain of greater than one is possible whilst keeping the information content of the data unchanged. This leads to a mathematical bound which only depends on the number of generated and training events. This study determines conditions on both the underlying and reconstructed probability distributions to ensure this bound. In particular, the resolution of variables in amplified data is not improved by the process but the increase in sample size can still improve statistical significance. The bound is confirmed using computer simulation and analysis of GAN generated data from the literature.