Noise as a Probe: Membership Inference Attacks on Diffusion Models Leveraging Initial Noise

TL;DR

This paper reveals that residual semantic signals in initial noise of diffusion models can be exploited to perform effective membership inference attacks, exposing privacy vulnerabilities.

Contribution

It introduces a novel MIA method leveraging residual semantic information in initial noise, without needing intermediate results or auxiliary datasets.

Findings

Residual semantics in initial noise reveal training data membership

The proposed attack outperforms existing methods in accuracy

Diffusion models are vulnerable to privacy attacks using this technique

Abstract

Diffusion models have achieved remarkable progress in image generation, but their increasing deployment raises serious concerns about privacy. In particular, fine-tuned models are highly vulnerable, as they are often fine-tuned on small and private datasets. Membership inference attacks (MIAs) are used to assess privacy risks by determining whether a specific sample was part of a model's training data. Existing MIAs against diffusion models either assume obtaining the intermediate results or require auxiliary datasets for training the shadow model. In this work, we utilized a critical yet overlooked vulnerability: the widely used noise schedules fail to fully eliminate semantic information in the images, resulting in residual semantic signals even at the maximum noise step. We empirically demonstrate that the fine-tuned diffusion model captures hidden correlations between the residual semantics in initial noise and the original images. Building on this insight, we propose a simple yet effective membership inference attack, which injects semantic information into the initial noise and infers membership by analyzing the model's generation result. Extensive experiments demonstrate that the semantic initial noise can strongly reveal membership information, highlighting the vulnerability of diffusion models to MIAs.