Semantic Mismatch and Perceptual Degradation: A New Perspective on Image Editing Immunity

TL;DR

This paper introduces a new perspective on image immunization against malicious diffusion-based edits, emphasizing semantic mismatch and perceptual degradation as key indicators of success, and proposes a novel method and metric to improve and evaluate immunization effectiveness.

Contribution

The paper proposes SIFM, a novel method that disrupts semantic alignment and induces perceptual degradation, along with ISR, a new metric to accurately measure immunization success.

Findings

SIFM achieves state-of-the-art immunization performance.

ISR effectively quantifies true immunization success.

Disrupting semantic and perceptual aspects enhances image protection.

Abstract

Text-guided image editing via diffusion models, while powerful, raises significant concerns about misuse, motivating efforts to immunize images against unauthorized edits using imperceptible perturbations. Prevailing metrics for evaluating immunization success typically rely on measuring the visual dissimilarity between the output generated from a protected image and a reference output generated from the unprotected original. This approach fundamentally overlooks the core requirement of image immunization, which is to disrupt semantic alignment with attacker intent, regardless of deviation from any specific output. We argue that immunization success should instead be defined by the edited output either semantically mismatching the prompt or suffering substantial perceptual degradations, both of which thwart malicious intent. To operationalize this principle, we propose Synergistic Intermediate Feature Manipulation (SIFM), a method that strategically perturbs intermediate diffusion features through dual synergistic objectives: (1) maximizing feature divergence from the original edit trajectory to disrupt semantic alignment with the expected edit, and (2) minimizing feature norms to induce perceptual degradations. Furthermore, we introduce the Immunization Success Rate (ISR), a novel metric designed to rigorously quantify true immunization efficacy for the first time. ISR quantifies the proportion of edits where immunization induces either semantic failure relative to the prompt or significant perceptual degradations, assessed via Multimodal Large Language Models (MLLMs). Extensive experiments show our SIFM achieves the state-of-the-art performance for safeguarding visual content against malicious diffusion-based manipulation.