Towards Reliable Identification of Diffusion-based Image Manipulations

TL;DR

This paper introduces RADAR, a new method for detecting and localizing diffusion-based image manipulations, achieving high accuracy and generalization across multiple models, supported by a new comprehensive benchmark.

Contribution

RADAR combines foundation model features with contrastive loss to improve detection of diffusion-based image edits and introduces BBC-PAIR, a new benchmark dataset.

Findings

RADAR outperforms state-of-the-art methods in detection accuracy.

RADAR generalizes well to unseen diffusion models.

BBC-PAIR provides a comprehensive evaluation platform.

Abstract

Changing facial expressions, gestures, or background details may dramatically alter the meaning conveyed by an image. Notably, recent advances in diffusion models greatly improve the quality of image manipulation while also opening the door to misuse. Identifying changes made to authentic images, thus, becomes an important task, constantly challenged by new diffusion-based editing tools. To this end, we propose a novel approach for ReliAble iDentification of inpainted AReas (RADAR). RADAR builds on existing foundation models and combines features from different image modalities. It also incorporates an auxiliary contrastive loss that helps to isolate manipulated image patches. We demonstrate these techniques to significantly improve both the accuracy of our method and its generalisation to a large number of diffusion models. To support realistic evaluation, we further introduce BBC-PAIR, a new comprehensive benchmark, with images tampered by 28 diffusion models. Our experiments show that RADAR achieves excellent results, outperforming the state-of-the-art in detecting and localising image edits made by both seen and unseen diffusion models. Our code, data and models will be publicly available at https://alex-costanzino.github.io/radar/.