G-PATE: Scalable Differentially Private Data Generator via Private Aggregation of Teacher Discriminators

TL;DR

G-PATE introduces a scalable differentially private data generator using private aggregation of teacher discriminators within a GAN framework, significantly improving privacy budget utilization and data utility, especially for high-dimensional image data.

Contribution

The paper presents G-PATE, a novel private GAN-based data generator with a private gradient aggregation mechanism ensuring differential privacy and high data utility under limited privacy budgets.

Findings

G-PATE outperforms prior methods in generating high-dimensional image data with strong privacy guarantees.

The private gradient aggregation effectively handles high-dimensional gradients.

G-PATE achieves high data utility with privacy budgets as low as ε ≤ 1.

Abstract

Recent advances in machine learning have largely benefited from the massive accessible training data. However, large-scale data sharing has raised great privacy concerns. In this work, we propose a novel privacy-preserving data Generative model based on the PATE framework (G-PATE), aiming to train a scalable differentially private data generator that preserves high generated data utility. Our approach leverages generative adversarial nets to generate data, combined with private aggregation among different discriminators to ensure strong privacy guarantees. Compared to existing approaches, G-PATE significantly improves the use of privacy budgets. In particular, we train a student data generator with an ensemble of teacher discriminators and propose a novel private gradient aggregation mechanism to ensure differential privacy on all information that flows from teacher discriminators to the student generator. In addition, with random projection and gradient discretization, the proposed gradient aggregation mechanism is able to effectively deal with high-dimensional gradient vectors. Theoretically, we prove that G-PATE ensures differential privacy for the data generator. Empirically, we demonstrate the superiority of G-PATE over prior work through extensive experiments. We show that G-PATE is the first work being able to generate high-dimensional image data with high data utility under limited privacy budgets ($\epsilon \le 1$). Our code is available at https://github.com/AI-secure/G-PATE.