Private Synthetic Data Generation in Bounded Memory

TL;DR

PrivHP is a lightweight, differentially private synthetic data generator that uses hierarchical decomposition and pruning to efficiently approximate data distributions within bounded memory, balancing privacy, utility, and space.

Contribution

It introduces PrivHP, a novel hierarchical, differentially private synthetic data generator with a tunable trade-off between space and utility, using pruning and private sketches.

Findings

Achieves differential privacy with space complexity O(k log^2 |X|).

Provides utility bounds considering hierarchy, noise, and pruning.

Ensures expected Wasserstein distance bounds from empirical distribution.

Abstract

We propose $\mathtt{PrivHP}$, a lightweight synthetic data generator with \textit{differential privacy} guarantees. $\mathtt{PrivHP}$ uses a novel hierarchical decomposition that approximates the input's cumulative distribution function (CDF) in bounded memory. It balances hierarchy depth, noise addition, and pruning of low-frequency subdomains while preserving frequent ones. Private sketches estimate subdomain frequencies efficiently without full data access. A key feature is the pruning parameter $k$, which controls the trade-off between space and utility. We define the skew measure $\mathtt{tail}_k$, capturing all but the top $k$ subdomain frequencies. Given a dataset $\mathcal{X}$, $\mathtt{PrivHP}$ uses $M=\mathcal{O}(k\log^2 |X|)$ space and, for input domain $\Omega = [0,1]$, ensures $\varepsilon$-differential privacy. It yields a generator with expected Wasserstein distance: \[ \mathcal{O}\left(\frac{\log^2 M}{\varepsilon n} + \frac{||\mathtt{tail}_k(\mathcal{X})||_1}{M n}\right) \] from the empirical distribution. This parameterized trade-off offers a level of flexibility unavailable in prior work. We also provide interpretable utility bounds that account for hierarchy depth, privacy noise, pruning, and frequency estimation errors.