On the Robustness of Latent Diffusion Models

TL;DR

This paper thoroughly analyzes the robustness of latent diffusion models against various attacks, evaluates their vulnerabilities in different scenarios, and provides a new benchmark dataset for future research.

Contribution

It offers a comprehensive robustness analysis of latent diffusion models, including white-box and black-box scenarios, and introduces a dataset for benchmarking defenses.

Findings

Identifies factors affecting model robustness.

Evaluates transfer attack effectiveness.

Provides a new dataset for robustness testing.

Abstract

Latent diffusion models achieve state-of-the-art performance on a variety of generative tasks, such as image synthesis and image editing. However, the robustness of latent diffusion models is not well studied. Previous works only focus on the adversarial attacks against the encoder or the output image under white-box settings, regardless of the denoising process. Therefore, in this paper, we aim to analyze the robustness of latent diffusion models more thoroughly. We first study the influence of the components inside latent diffusion models on their white-box robustness. In addition to white-box scenarios, we evaluate the black-box robustness of latent diffusion models via transfer attacks, where we consider both prompt-transfer and model-transfer settings and possible defense mechanisms. However, all these explorations need a comprehensive benchmark dataset, which is missing in the literature. Therefore, to facilitate the research of the robustness of latent diffusion models, we propose two automatic dataset construction pipelines for two kinds of image editing models and release the whole dataset. Our code and dataset are available at \url{https://github.com/jpzhang1810/LDM-Robustness}.