SImProv: Scalable Image Provenance Framework for Robust Content Attribution

TL;DR

SImProv is a scalable framework for image provenance that efficiently retrieves, verifies, and localizes manipulations in images, demonstrating robustness to common transformations and achieving high accuracy on large datasets.

Contribution

It introduces a novel end-to-end trainable warping module for robust image comparison and a scalable system capable of handling 100 million images for provenance and manipulation detection.

Findings

Effective retrieval over 100 million images.

Robust manipulation localization despite common transformations.

End-to-end training improves comparison accuracy.

Abstract

We present SImProv - a scalable image provenance framework to match a query image back to a trusted database of originals and identify possible manipulations on the query. SImProv consists of three stages: a scalable search stage for retrieving top-k most similar images; a re-ranking and near-duplicated detection stage for identifying the original among the candidates; and finally a manipulation detection and visualization stage for localizing regions within the query that may have been manipulated to differ from the original. SImProv is robust to benign image transformations that commonly occur during online redistribution, such as artifacts due to noise and recompression degradation, as well as out-of-place transformations due to image padding, warping, and changes in size and shape. Robustness towards out-of-place transformations is achieved via the end-to-end training of a differentiable warping module within the comparator architecture. We demonstrate effective retrieval and manipulation detection over a dataset of 100 million images.