A Combinatorial Approach to Synthetic Data Generation for Machine Learning

TL;DR

This paper introduces a new method for generating synthetic data that reduces the number of samples needed while maintaining model performance and improving privacy.

Contribution

The novel combinatorial sampling approach reduces sample size requirements and preserves model accuracy with fewer data points.

Findings

Combinatorial sampling achieves comparable model performance with fewer synthetic samples than random sampling.

The method maintains better accuracy when combined with differential privacy compared to traditional approaches.

Model predictions are often influenced more by feature interactions than by using all features.

Abstract

Datasets used in machine learning often contain sensitive information, including personally identifiable health and financial details. A common challenge faced by organizations and researchers is the risk of privacy breaches when using real-world data. Synthetic data can be used as an alternative to the real-world data. In existing synthetic data generation techniques, an encoder processes the real-world data to map it into a lower-dimensional latent space. Random sampling is then performed in this latent space. Subsequently, a decoder network is utilized to generate synthetic data from these sampled points in the latent space. Such approaches typically require generating a large number of synthetic samples to approximate the performance of real-world data, subsequently slowing down downstream machine learning tasks. Addressing this, we introduce a combinatorial approach to sampling the latent space, motivated by our empirical findings within this study that most model predictions are largely influenced by interactions between a few features. In some cases, just using a small number of features produces accuracy better than using entire features. Through this approach, we generate samples that utilize t-way interactions among the t latent dimensions out of n. Our experimental results indicate that our approach requires fewer samples than traditional random sampling to achieve comparable model performance for real-world data sets. We also show that when integrated with a differentially private mechanism, our approach incurs a smaller decline in model performance than existing random sampling approach.

Figures