Creating Blank Canvas Against AI-enabled Image Forgery

TL;DR

This paper proposes a novel tampering detection method using the Segment Anything Model (SAM) by transforming images into a blank canvas and applying frequency-aware adversarial perturbations, effectively identifying forged regions.

Contribution

It introduces a new approach that leverages SAM and adversarial perturbations to detect image forgeries without retraining the model, enhancing tamper localization.

Findings

Effective detection of forged regions demonstrated

Frequency-aware optimization improves tamper localization

Method outperforms existing detection techniques

Abstract

AIGC-based image editing technology has greatly simplified the realistic-level image modification, causing serious potential risks of image forgery. This paper introduces a new approach to tampering detection using the Segment Anything Model (SAM). Instead of training SAM to identify tampered areas, we propose a novel strategy. The entire image is transformed into a blank canvas from the perspective of neural models. Any modifications to this blank canvas would be noticeable to the models. To achieve this idea, we introduce adversarial perturbations to prevent SAM from ``seeing anything'', allowing it to identify forged regions when the image is tampered with. Due to SAM's powerful perceiving capabilities, naive adversarial attacks cannot completely tame SAM. To thoroughly deceive SAM and make it blind to the image, we introduce a frequency-aware optimization strategy, which further enhances the capability of tamper localization. Extensive experimental results demonstrate the effectiveness of our method.