A Unified View of Differentially Private Deep Generative Modeling

TL;DR

This paper provides a comprehensive framework that unifies various methods for differentially private deep generative modeling, highlighting their strengths, limitations, and guiding future research directions.

Contribution

It introduces a systematic view that consolidates existing approaches, facilitating the design of privacy-preserving data generation methods for diverse applications.

Findings

Unified framework for DP deep generative models

Analysis of strengths and limitations of existing approaches

Guidance for future research in privacy-preserving data generation

Abstract

The availability of rich and vast data sources has greatly advanced machine learning applications in various domains. However, data with privacy concerns comes with stringent regulations that frequently prohibited data access and data sharing. Overcoming these obstacles in compliance with privacy considerations is key for technological progress in many real-world application scenarios that involve privacy sensitive data. Differentially private (DP) data publishing provides a compelling solution, where only a sanitized form of the data is publicly released, enabling privacy-preserving downstream analysis and reproducible research in sensitive domains. In recent years, various approaches have been proposed for achieving privacy-preserving high-dimensional data generation by private training on top of deep neural networks. In this paper, we present a novel unified view that systematizes these approaches. Our view provides a joint design space for systematically deriving methods that cater to different use cases. We then discuss the strengths, limitations, and inherent correlations between different approaches, aiming to shed light on crucial aspects and inspire future research. We conclude by presenting potential paths forward for the field of DP data generation, with the aim of steering the community toward making the next important steps in advancing privacy-preserving learning.