DPDSyn: Improving Differentially Private Dataset Synthesis for Model Training by Downstream Task Guidance

TL;DR

DPDSyn introduces a novel approach to differentially private dataset synthesis by leveraging trained models for downstream tasks, significantly enhancing data utility and efficiency.

Contribution

The paper proposes using differentially private models trained on original data to guide synthetic data generation, improving utility over existing low-dimensional distribution methods.

Findings

DPDSyn outperforms eight baselines in accuracy by up to 2.40x.

DPDSyn achieves up to 333.73x better synthesis efficiency.

The method demonstrates strong scalability across different data scales.

Abstract

How to synthesize a dataset while achieving differential privacy for AI model training is a meaningful but challenging problem. To address this problem, state-of-the-art methods first select original private dataset's multiple low-dimensional distributions that have the potential to approximate the distribution of original private dataset with high precision, and then synthesize a dataset obeying all selected low-dimensional distributions as the synthetic dataset. However, it is difficult to select suitable low-dimensional distributions, which in turn degrades the data utility of resulting synthetic dataset. To improve differentially private dataset synthesis, we propose to train a differentially private AI model for downstream tasks on the original private dataset and utilize the trained model to synthesize datasets. In particular, on the one hand, the AI model satisfies differential privacy so no matter how to use the model does not disclose private information of original private dataset. On the other hand, the AI model is trained to complete the downstream task so the AI model preserves critical information for completing downstream tasks. We utilize the AI model to synthesize datasets to achieve the goal of improving data utility while preserving privacy. Empirical evaluations on four benchmark datasets demonstrate that our proposed DPDSyn consistently outperforms eight state-of-the-art baselines with a maximum improvement of 2.40x in accuracy and 333.73x in synthesis efficiency. Further experiments also validate that DPDSyn has strong scalability across varying data scales.