Synthetic Data: Revisiting the Privacy-Utility Trade-off

TL;DR

This paper critically examines claims that synthetic data does not improve privacy-utility trade-offs over traditional methods, and finds that previous refutations were based on constrained environments, reaffirming synthetic data's advantages.

Contribution

The study validates the privacy-utility benefits of synthetic data in general settings and clarifies misconceptions caused by prior specialized analyses.

Findings

Synthetic data offers better privacy-utility trade-offs than k-anonymization.

Previous claims of privacy breaches in PATE-GAN and PrivBayes were due to constrained experimental setups.

In general environments, synthetic data maintains differential privacy guarantees and utility benefits.

Abstract

Synthetic data has been considered a better privacy-preserving alternative to traditionally sanitized data across various applications. However, a recent article challenges this notion, stating that synthetic data does not provide a better trade-off between privacy and utility than traditional anonymization techniques, and that it leads to unpredictable utility loss and highly unpredictable privacy gain. The article also claims to have identified a breach in the differential privacy guarantees provided by PATE-GAN and PrivBayes. When a study claims to refute or invalidate prior findings, it is crucial to verify and validate the study. In our work, we analyzed the implementation of the privacy game described in the article and found that it operated in a highly specialized and constrained environment, which limits the applicability of its findings to general cases. Our exploration also revealed that the game did not satisfy a crucial precondition concerning data distributions, which contributed to the perceived violation of the differential privacy guarantees offered by PATE-GAN and PrivBayes. We also conducted a privacy-utility trade-off analysis in a more general and unconstrained environment. Our experimentation demonstrated that synthetic data indeed achieves a more favorable privacy-utility trade-off compared to the provided implementation of k-anonymization, thereby reaffirming earlier conclusions.