Adaptive Forensic Feature Refinement via Intrinsic Importance Perception

TL;DR

This paper introduces I2P, a novel SID framework that adaptively identifies critical feature layers and constrains updates to preserve pretrained structure, enhancing cross-distribution generalization in synthetic image detection.

Contribution

The paper proposes a joint optimization approach for VFM adaptation in SID, explicitly modeling the optimal representational hierarchy and constraining parameter updates.

Findings

I2P effectively identifies the most discriminative feature layers for SID.

The method improves generalization to unknown generation sources.

Experimental results demonstrate enhanced detection accuracy across diverse datasets.

Abstract

With the rapid development of generative models and multimodal content editing technologies, the key challenge faced by synthetic image detection (SID) lies in cross-distribution generalization to unknown generation sources. In recent years, visual foundation models (VFM), which acquire rich visual priors through large scale image-text alignment pretraining, have become a promising technical route for improving the generalization ability of SID. However, existing VFM-based methods remain relatively coarse-grained in their adaptation strategies. They typically either directly use the final layer representations of VFM or simply fuse multi layer features, lacking explicit modeling of the optimal representational hierarchy for transferable forgery cues. Meanwhile, although directly fine-tuning VFM can enhance task adaptation, it may also damage the cross-modal pretrained structure that supports open-set generalization. To address this task specific tension, we reformulate VFM adaptation for SID as a joint optimization problem: it is necessary both to identify the critical representational layer that is more suitable for carrying forgery discriminative information and to constrain the disturbance caused by task knowledge injection to the pretrained structure. Based on this, we propose I2P, an SID framework centered on intrinsic importance perception. I2P first adaptively identifies the critical layer representations that are most discriminative for SID, and then constrains task-driven parameter updates within a low sensitivity parameter subspace, thereby improving task specificity while preserving the transferable structure of pretrained representations as much as possible.