Constrained R-CNN: A general image manipulation detection model

TL;DR

Constrained R-CNN is a comprehensive image manipulation detection model that combines data-driven feature learning, attention mechanisms, and multi-level feature fusion to improve localization and classification accuracy, achieving state-of-the-art results.

Contribution

It introduces a unified, learnable feature extractor and a multi-stage architecture that jointly performs manipulation localization and classification, enhancing universality and accuracy.

Findings

Achieves state-of-the-art performance on multiple datasets.

F1 score improvements of 28.4%, 73.2%, and 13.3%.

Effectively discriminates manipulated regions with attention mechanisms.

Abstract

Recently, deep learning-based models have exhibited remarkable performance for image manipulation detection. However, most of them suffer from poor universality of handcrafted or predetermined features. Meanwhile, they only focus on manipulation localization and overlook manipulation classification. To address these issues, we propose a coarse-to-fine architecture named Constrained R-CNN for complete and accurate image forensics. First, the learnable manipulation feature extractor learns a unified feature representation directly from data. Second, the attention region proposal network effectively discriminates manipulated regions for the next manipulation classification and coarse localization. Then, the skip structure fuses low-level and high-level information to refine the global manipulation features. Finally, the coarse localization information guides the model to further learn the finer local features and segment out the tampered region. Experimental results show that our model achieves state-of-the-art performance. Especially, the F1 score is increased by 28.4%, 73.2%, 13.3% on the NIST16, COVERAGE, and Columbia dataset.