X-Edit: Detecting and Localizing Edits in Images Altered by Text-Guided Diffusion Models

TL;DR

X-Edit is a novel method that accurately localizes diffusion-based edits in images, addressing a critical need for detecting subtle deepfake modifications caused by text-guided diffusion models.

Contribution

The paper introduces X-Edit, the first approach to localize diffusion-based image edits, along with a new dataset for this task and an effective segmentation and relevance loss training strategy.

Findings

X-Edit outperforms baselines in PSNR and SSIM metrics.

The method effectively localizes subtle diffusion-based edits.

The approach enhances forensic detection of advanced image manipulations.

Abstract

Text-guided diffusion models have significantly advanced image editing, enabling highly realistic and local modifications based on textual prompts. While these developments expand creative possibilities, their malicious use poses substantial challenges for detection of such subtle deepfake edits. To this end, we introduce Explain Edit (X-Edit), a novel method for localizing diffusion-based edits in images. To localize the edits for an image, we invert the image using a pretrained diffusion model, then use these inverted features as input to a segmentation network that explicitly predicts the edited masked regions via channel and spatial attention. Further, we finetune the model using a combined segmentation and relevance loss. The segmentation loss ensures accurate mask prediction by balancing pixel-wise errors and perceptual similarity, while the relevance loss guides the model to focus on low-frequency regions and mitigate high-frequency artifacts, enhancing the localization of subtle edits. To the best of our knowledge, we are the first to address and model the problem of localizing diffusion-based modified regions in images. We additionally contribute a new dataset of paired original and edited images addressing the current lack of resources for this task. Experimental results demonstrate that X-Edit accurately localizes edits in images altered by text-guided diffusion models, outperforming baselines in PSNR and SSIM metrics. This highlights X-Edit's potential as a robust forensic tool for detecting and pinpointing manipulations introduced by advanced image editing techniques.