Semantic Discrepancy-aware Detector for Image Forgery Identification

TL;DR

This paper introduces a Semantic Discrepancy-aware Detector (SDD) that uses reconstruction learning and semantic token sampling to improve image forgery detection by aligning semantic and visual spaces, resulting in superior performance.

Contribution

The paper proposes a novel SDD method that leverages pre-trained vision language models and a concept-level discrepancy learning module to enhance forgery detection accuracy.

Findings

SDD outperforms existing methods on standard datasets.

Semantic token sampling reduces space shift effects.

Concept-level discrepancy learning improves forgery trace capture.

Abstract

With the rapid advancement of image generation techniques, robust forgery detection has become increasingly imperative to ensure the trustworthiness of digital media. Recent research indicates that the learned semantic concepts of pre-trained models are critical for identifying fake images. However, the misalignment between the forgery and semantic concept spaces hinders the model's forgery detection performance. To address this problem, we propose a novel Semantic Discrepancy-aware Detector (SDD) that leverages reconstruction learning to align the two spaces at a fine-grained visual level. By exploiting the conceptual knowledge embedded in the pre-trained vision language model, we specifically design a semantic token sampling module to mitigate the space shifts caused by features irrelevant to both forgery traces and semantic concepts. A concept-level forgery discrepancy learning module, built upon a visual reconstruction paradigm, is proposed to strengthen the interaction between visual semantic concepts and forgery traces, effectively capturing discrepancies under the concepts' guidance. Finally, the low-level forgery feature enhancemer integrates the learned concept level forgery discrepancies to minimize redundant forgery information. Experiments conducted on two standard image forgery datasets demonstrate the efficacy of the proposed SDD, which achieves superior results compared to existing methods. The code is available at https://github.com/wzy1111111/SSD.