Exploring Multi-view Pixel Contrast for General and Robust Image Forgery Localization

TL;DR

This paper introduces a Multi-view Pixel-wise Contrastive (MPC) method for image forgery localization that improves pixel relationship modeling through contrastive pre-training, leading to better generalization and robustness.

Contribution

The paper proposes a novel contrastive learning approach for pixel relationship modeling in forgery localization, enhancing performance over existing methods.

Findings

MPC outperforms state-of-the-art methods on multiple datasets.

The contrastive pre-training improves robustness against post-processing.

The approach generalizes well across different scales and modalities.

Abstract

Image forgery localization, which aims to segment tampered regions in an image, is a fundamental yet challenging digital forensic task. While some deep learning-based forensic methods have achieved impressive results, they directly learn pixel-to-label mappings without fully exploiting the relationship between pixels in the feature space. To address such deficiency, we propose a Multi-view Pixel-wise Contrastive algorithm (MPC) for image forgery localization. Specifically, we first pre-train the backbone network with the supervised contrastive loss to model pixel relationships from the perspectives of within-image, cross-scale and cross-modality. That is aimed at increasing intra-class compactness and inter-class separability. Then the localization head is fine-tuned using the cross-entropy loss, resulting in a better pixel localizer. The MPC is trained on three different scale training datasets to make a comprehensive and fair comparison with existing image forgery localization algorithms. Extensive experiments on the small, medium and large scale training datasets show that the proposed MPC achieves higher generalization performance and robustness against post-processing than the state-of-the-arts. Code will be available at https://github.com/multimediaFor/MPC.