Anti-Inpainting: A Proactive Defense Approach against Malicious Diffusion-based Inpainters under Unknown Conditions

TL;DR

Anti-Inpainting introduces a proactive defense method against diffusion-based image tampering, utilizing multi-level feature extraction, semantic-preserving augmentation, and distribution deviation optimization to enhance robustness under unknown conditions.

Contribution

It presents a novel multi-module approach combining feature extraction, data augmentation, and optimization strategies for proactive image tampering defense.

Findings

Effective against diffusion-based inpainters under unknown conditions.

Robust against image purification methods.

Transferable across different diffusion model versions.

Abstract

With the increasing prevalence of diffusion-based malicious image manipulation, existing proactive defense methods struggle to safeguard images against tampering under unknown conditions. To address this, we propose Anti-Inpainting, a proactive defense approach that achieves protection comprising three novel modules. First, we introduce a multi-level deep feature extractor to obtain intricate features from the diffusion denoising process, enhancing protective effectiveness. Second, we design a multi-scale, semantic-preserving data augmentation technique to enhance the transferability of adversarial perturbations across unknown conditions. Finally, we propose a selection-based distribution deviation optimization strategy to bolster protection against manipulations guided by diverse random seeds. Extensive experiments on InpaintGuardBench and CelebA-HQ demonstrate that Anti-Inpainting effectively defends against diffusion-based inpainters under unknown conditions. Additionally, our approach demonstrates robustness against various image purification methods and transferability across different diffusion model versions.