Towards Generic Image Manipulation Detection with Weakly-Supervised Self-Consistency Learning

TL;DR

This paper introduces a weakly-supervised learning approach for image manipulation detection that requires only image-level labels, improving generalization and localization without pixel-level annotations.

Contribution

It proposes a novel weakly-supervised self-consistency learning framework leveraging multi-source and inter-patch consistency for better manipulation detection.

Findings

Competitive performance with fully-supervised methods

Effective in out-of-distribution scenarios

Reasonable localization of manipulated regions

Abstract

As advanced image manipulation techniques emerge, detecting the manipulation becomes increasingly important. Despite the success of recent learning-based approaches for image manipulation detection, they typically require expensive pixel-level annotations to train, while exhibiting degraded performance when testing on images that are differently manipulated compared with training images. To address these limitations, we propose weakly-supervised image manipulation detection, such that only binary image-level labels (authentic or tampered with) are required for training purpose. Such a weakly-supervised setting can leverage more training images and has the potential to adapt quickly to new manipulation techniques. To improve the generalization ability, we propose weakly-supervised self-consistency learning (WSCL) to leverage the weakly annotated images. Specifically, two consistency properties are learned: multi-source consistency (MSC) and inter-patch consistency (IPC). MSC exploits different content-agnostic information and enables cross-source learning via an online pseudo label generation and refinement process. IPC performs global pair-wise patch-patch relationship reasoning to discover a complete region of manipulation. Extensive experiments validate that our WSCL, even though is weakly supervised, exhibits competitive performance compared with fully-supervised counterpart under both in-distribution and out-of-distribution evaluations, as well as reasonable manipulation localization ability.