Provenance Filtering for Multimedia Phylogeny

TL;DR

This paper introduces a two-tiered provenance filtering method for multimedia phylogeny that effectively identifies related images and their contributions, aiding digital forensics in analyzing image evolution and manipulation.

Contribution

It proposes a novel two-tiered approach combining coarse and refined searches to identify potential source images and their parts in multimedia phylogeny.

Findings

High accuracy in identifying related images in large datasets

Effective detection of composite and doctored images

Scalability demonstrated on over a million images

Abstract

Departing from traditional digital forensics modeling, which seeks to analyze single objects in isolation, multimedia phylogeny analyzes the evolutionary processes that influence digital objects and collections over time. One of its integral pieces is provenance filtering, which consists of searching a potentially large pool of objects for the most related ones with respect to a given query, in terms of possible ancestors (donors or contributors) and descendants. In this paper, we propose a two-tiered provenance filtering approach to find all the potential images that might have contributed to the creation process of a given query $q$. In our solution, the first (coarse) tier aims to find the most likely "host" images --- the major donor or background --- contributing to a composite/doctored image. The search is then refined in the second tier, in which we search for more specific (potentially small) parts of the query that might have been extracted from other images and spliced into the query image. Experimental results with a dataset containing more than a million images show that the two-tiered solution underpinned by the context of the query is highly useful for solving this difficult task.