TrainFors: A Large Benchmark Training Dataset for Image Manipulation Detection and Localization

TL;DR

This paper introduces TrainFors, a standardized large-scale training dataset for image manipulation detection and localization, aiming to enable fairer comparisons of different methods.

Contribution

The authors create and release a comprehensive benchmark training dataset for various image forgery types, addressing inconsistencies in existing datasets and providing a basis for fair evaluation.

Findings

State-of-the-art IMDL methods trained on TrainFors show different performance levels.

Identified issues in existing datasets and proposed necessary modifications.

Provided a fair comparison framework for IMDL research.

Abstract

The evaluation datasets and metrics for image manipulation detection and localization (IMDL) research have been standardized. But the training dataset for such a task is still nonstandard. Previous researchers have used unconventional and deviating datasets to train neural networks for detecting image forgeries and localizing pixel maps of manipulated regions. For a fair comparison, the training set, test set, and evaluation metrics should be persistent. Hence, comparing the existing methods may not seem fair as the results depend heavily on the training datasets as well as the model architecture. Moreover, none of the previous works release the synthetic training dataset used for the IMDL task. We propose a standardized benchmark training dataset for image splicing, copy-move forgery, removal forgery, and image enhancement forgery. Furthermore, we identify the problems with the existing IMDL datasets and propose the required modifications. We also train the state-of-the-art IMDL methods on our proposed TrainFors1 dataset for a fair evaluation and report the actual performance of these methods under similar conditions.