Noise Doesn't Lie: Towards Universal Detection of Deep Inpainting

TL;DR

This paper introduces a universal deep inpainting detection method that leverages noise pattern discrepancies and a novel training dataset, significantly improving detection accuracy and generalization across various inpainting techniques.

Contribution

The paper presents a new universal detection framework and a noise-based training dataset that enhance the detection of diverse deep inpainting methods.

Findings

Outperforms existing detection methods on multiple benchmarks.

Generalizes well to unseen inpainting techniques.

Universal training dataset boosts other detection methods.

Abstract

Deep image inpainting aims to restore damaged or missing regions in an image with realistic contents. While having a wide range of applications such as object removal and image recovery, deep inpainting techniques also have the risk of being manipulated for image forgery. A promising countermeasure against such forgeries is deep inpainting detection, which aims to locate the inpainted regions in an image. In this paper, we make the first attempt towards universal detection of deep inpainting, where the detection network can generalize well when detecting different deep inpainting methods. To this end, we first propose a novel data generation approach to generate a universal training dataset, which imitates the noise discrepancies exist in real versus inpainted image contents to train universal detectors. We then design a Noise-Image Cross-fusion Network (NIX-Net) to effectively exploit the discriminative information contained in both the images and their noise patterns. We empirically show, on multiple benchmark datasets, that our approach outperforms existing detection methods by a large margin and generalize well to unseen deep inpainting techniques. Our universal training dataset can also significantly boost the generalizability of existing detection methods.