Auto-Focus Contrastive Learning for Image Manipulation Detection

TL;DR

This paper introduces AF-CL, a novel contrastive learning approach that enhances image manipulation detection by focusing on manipulated regions and modeling trace relations, leading to improved accuracy over existing methods.

Contribution

The paper proposes a new AF-CL network with multi-scale view generation and trace relation modeling to better detect manipulated regions in images.

Findings

Achieves up to 2.5% F1 score improvement on CAISA dataset

Outperforms state-of-the-art methods on NIST and Coverage datasets

Effectively focuses on manipulated regions and explores trace relations

Abstract

Generally, current image manipulation detection models are simply built on manipulation traces. However, we argue that those models achieve sub-optimal detection performance as it tends to: 1) distinguish the manipulation traces from a lot of noisy information within the entire image, and 2) ignore the trace relations among the pixels of each manipulated region and its surroundings. To overcome these limitations, we propose an Auto-Focus Contrastive Learning (AF-CL) network for image manipulation detection. It contains two main ideas, i.e., multi-scale view generation (MSVG) and trace relation modeling (TRM). Specifically, MSVG aims to generate a pair of views, each of which contains the manipulated region and its surroundings at a different scale, while TRM plays a role in modeling the trace relations among the pixels of each manipulated region and its surroundings for learning the discriminative representation. After learning the AF-CL network by minimizing the distance between the representations of corresponding views, the learned network is able to automatically focus on the manipulated region and its surroundings and sufficiently explore their trace relations for accurate manipulation detection. Extensive experiments demonstrate that, compared to the state-of-the-arts, AF-CL provides significant performance improvements, i.e., up to 2.5%, 7.5%, and 0.8% F1 score, on CAISA, NIST, and Coverage datasets, respectively.