Differentially Private Diffusion Models

TL;DR

This paper introduces Differentially Private Diffusion Models (DPDMs) that leverage diffusion models with differential privacy to generate high-quality synthetic data, achieving state-of-the-art results and matching the performance of private classifiers trained directly on sensitive data.

Contribution

The paper presents a novel DP training method for diffusion models, including noise multiplicity, and demonstrates superior performance in image generation and classification tasks.

Findings

Achieved state-of-the-art results on image generation benchmarks.

Classifiers trained on DPDM synthetic data perform comparably to DP-trained classifiers.

Proposed noise multiplicity enhances DP training of diffusion models.

Abstract

While modern machine learning models rely on increasingly large training datasets, data is often limited in privacy-sensitive domains. Generative models trained with differential privacy (DP) on sensitive data can sidestep this challenge, providing access to synthetic data instead. We build on the recent success of diffusion models (DMs) and introduce Differentially Private Diffusion Models (DPDMs), which enforce privacy using differentially private stochastic gradient descent (DP-SGD). We investigate the DM parameterization and the sampling algorithm, which turn out to be crucial ingredients in DPDMs, and propose noise multiplicity, a powerful modification of DP-SGD tailored to the training of DMs. We validate our novel DPDMs on image generation benchmarks and achieve state-of-the-art performance in all experiments. Moreover, on standard benchmarks, classifiers trained on DPDM-generated synthetic data perform on par with task-specific DP-SGD-trained classifiers, which has not been demonstrated before for DP generative models. Project page and code: https://nv-tlabs.github.io/DPDM.