# An "On The Fly" Framework for Efficiently Generating Synthetic Big Data Sets

## TL;DR

This paper introduces an efficient 'On the Fly' framework for generating large synthetic data sets, enhancing data analytics by addressing computational challenges and supporting diverse applications.

## Contribution

The paper presents a novel real-time synthetic data generation framework that improves efficiency and versatility for large-scale data analytics tasks.

## Key findings

- Framework is computationally efficient
- Applicable to diverse data generation problems
- Mathematical analysis confirms effectiveness

## Abstract

Collecting, analyzing and gaining insight from large volumes of data is now the norm in an ever increasing number of industries. Data analytics techniques, such as machine learning, are powerful tools used to analyze these large volumes of data. Synthetic data sets are routinely relied upon to train and develop such data analytics methods for several reasons: to generate larger data sets than are available, to generate diverse data sets, to preserve anonymity in data sets with sensitive information, etc. Processing, transmitting and storing data is a key issue faced when handling large data sets. This paper presents an "On the fly" framework for generating big synthetic data sets, suitable for these data analytics methods, that is both computationally efficient and applicable to a diverse set of problems. An example application of the proposed framework is presented along with a mathematical analysis of its computational efficiency, demonstrating its effectiveness.