Membership Inference Attacks against Diffusion Models

TL;DR

This paper investigates the privacy vulnerabilities of diffusion models against membership inference attacks, comparing their resistance to GANs and analyzing the effects of hyperparameters like time steps and sampling steps.

Contribution

It provides the first comprehensive analysis of diffusion models' resistance to membership inference attacks and highlights the significance of hyperparameters in privacy leakage.

Findings

Diffusion models show comparable resistance to GANs in membership inference attacks.

Intermediate noise schedule steps are most vulnerable to attacks.

Sampling steps significantly affect model privacy, while sampling variances have limited impact.

Abstract

Diffusion models have attracted attention in recent years as innovative generative models. In this paper, we investigate whether a diffusion model is resistant to a membership inference attack, which evaluates the privacy leakage of a machine learning model. We primarily discuss the diffusion model from the standpoints of comparison with a generative adversarial network (GAN) as conventional models and hyperparameters unique to the diffusion model, i.e., time steps, sampling steps, and sampling variances. We conduct extensive experiments with DDIM as a diffusion model and DCGAN as a GAN on the CelebA and CIFAR-10 datasets in both white-box and black-box settings and then confirm if the diffusion model is comparably resistant to a membership inference attack as GAN. Next, we demonstrate that the impact of time steps is significant and intermediate steps in a noise schedule are the most vulnerable to the attack. We also found two key insights through further analysis. First, we identify that DDIM is vulnerable to the attack for small sample sizes instead of achieving a lower FID. Second, sampling steps in hyperparameters are important for resistance to the attack, whereas the impact of sampling variances is quite limited.