Private Synthetic Text Generation with Diffusion Models

TL;DR

This paper investigates the effectiveness of diffusion models for generating private synthetic texts under differential privacy, comparing their performance to open-source LLMs and critically examining previous methods.

Contribution

It provides the first extensive experimental evaluation of diffusion models for private text generation and reanalyzes prior private LLM approaches for privacy guarantee violations.

Findings

Diffusion models underperform compared to open-source LLMs in private text generation.

Previous private LLM works may have violated differential privacy guarantees.

Open-source LLMs outperform diffusion models in the privacy regime.

Abstract

How capable are diffusion models of generating synthetics texts? Recent research shows their strengths, with performance reaching that of auto-regressive LLMs. But are they also good in generating synthetic data if the training was under differential privacy? Here the evidence is missing, yet the promises from private image generation look strong. In this paper we address this open question by extensive experiments. At the same time, we critically assess (and reimplement) previous works on synthetic private text generation with LLMs and reveal some unmet assumptions that might have led to violating the differential privacy guarantees. Our results partly contradict previous non-private findings and show that fully open-source LLMs outperform diffusion models in the privacy regime. Our complete source codes, datasets, and experimental setup is publicly available to foster future research.