Investigating Human Factors in Image Forgery Detection

TL;DR

This paper investigates human factors in image forgery detection using eye-tracking, compares human and automated performance, and develops a model to predict image difficulty for forgery detection.

Contribution

It provides a subjective evaluation with eye-tracking data, compares human and algorithmic detection, and introduces a model to predict image difficulty levels.

Findings

Humans outperform automated algorithms in forgery detection.

Eye-tracking reveals key human factors influencing detection.

A predictive model estimates image difficulty for forgery detection.

Abstract

In today's age of internet and social media, one can find an enormous volume of forged images on-line. These images have been used in the past to convey falsified information and achieve harmful intentions. The spread and the effect of the social media only makes this problem more severe. While creating forged images has become easier due to software advancements, there is no automated algorithm which can reliably detect forgery. Image forgery detection can be seen as a subset of image understanding problem. Human performance is still the gold-standard for these type of problems when compared to existing state-of-art automated algorithms. We conduct a subjective evaluation test with the aid of eye-tracker to investigate into human factors associated with this problem. We compare the performance of an automated algorithm and humans for forgery detection problem. We also develop an algorithm which uses the data from the evaluation test to predict the difficulty-level of an image (the difficulty-level of an image here denotes how difficult it is for humans to detect forgery in an image. Terms such as "Easy/difficult image" will be used in the same context). The experimental results presented in this paper should facilitate development of better algorithms in the future.