Towards Black-Box Membership Inference Attack for Diffusion Models

TL;DR

This paper presents a novel black-box membership inference attack on diffusion models that does not require internal model access, effectively identifying training data membership using only API calls and output comparisons.

Contribution

The work introduces a new black-box MIA method for diffusion models that operates solely via API, outperforming existing techniques and extending to Diffusion Transformer architectures.

Findings

Outperforms previous MIA methods on DDIM and Stable Diffusion

Effective in black-box setting without internal model access

Extends to Diffusion Transformer architecture

Abstract

Given the rising popularity of AI-generated art and the associated copyright concerns, identifying whether an artwork was used to train a diffusion model is an important research topic. The work approaches this problem from the membership inference attack (MIA) perspective. We first identify the limitation of applying existing MIA methods for proprietary diffusion models: the required access of internal U-nets. To address the above problem, we introduce a novel membership inference attack method that uses only the image-to-image variation API and operates without access to the model's internal U-net. Our method is based on the intuition that the model can more easily obtain an unbiased noise prediction estimate for images from the training set. By applying the API multiple times to the target image, averaging the outputs, and comparing the result to the original image, our approach can classify whether a sample was part of the training set. We validate our method using DDIM and Stable Diffusion setups and further extend both our approach and existing algorithms to the Diffusion Transformer architecture. Our experimental results consistently outperform previous methods.