Graphical vs. Deep Generative Models: Measuring the Impact of Differentially Private Mechanisms and Budgets on Utility

TL;DR

This study compares graphical and deep generative models with differential privacy, analyzing how privacy budgets affect utility and providing insights for selecting suitable models based on dataset features and privacy requirements.

Contribution

It provides a detailed measurement study of how DP mechanisms allocate privacy budgets in graphical and deep generative models for tabular data, highlighting their strengths and limitations.

Findings

Graphical models distribute privacy budgets horizontally, limiting their use on wide datasets.

Deep models' privacy budget spending varies per iteration, offering more flexibility.

Higher privacy levels (ε≥100) can improve model generalization and utility.

Abstract

Generative models trained with Differential Privacy (DP) can produce synthetic data while reducing privacy risks. However, navigating their privacy-utility tradeoffs makes finding the best models for specific settings/tasks challenging. This paper bridges this gap by profiling how DP generative models for tabular data distribute privacy budgets across rows and columns, which is one of the primary sources of utility degradation. We compare graphical and deep generative models, focusing on the key factors contributing to how privacy budgets are spent, i.e., underlying modeling techniques, DP mechanisms, and data dimensionality. Through our measurement study, we shed light on the characteristics that make different models suitable for various settings and tasks. For instance, we find that graphical models distribute privacy budgets horizontally and thus cannot handle relatively wide datasets for a fixed training time; also, the performance on the task they were optimized for monotonically increases with more data but could also overfit. Deep generative models spend their budgets per iteration, so their behavior is less predictable with varying dataset dimensions, but are more flexible as they could perform better if trained on more features. Moreover, low levels of privacy ($\epsilon\geq100$) could help some models generalize, achieving better results than without applying DP. We believe our work will aid the deployment of DP synthetic data techniques by navigating through the best candidate models vis-a-vis the dataset features, desired privacy levels, and downstream tasks.