InpDiffusion: Image Inpainting Localization via Conditional Diffusion Models

TL;DR

This paper introduces InpDiffusion, a diffusion model-based approach for image inpainting localization that improves boundary detection and reduces overconfidence through semantic and edge condition integration.

Contribution

The paper presents a novel diffusion model framework with edge supervision and a dual-stream multi-scale feature extractor for enhanced inpainting localization.

Findings

Outperforms state-of-the-art IIL methods on challenging datasets.

Demonstrates robustness and generalization in various scenarios.

Effectively detects subtle tampering boundaries.

Abstract

As artificial intelligence advances rapidly, particularly with the advent of GANs and diffusion models, the accuracy of Image Inpainting Localization (IIL) has become increasingly challenging. Current IIL methods face two main challenges: a tendency towards overconfidence, leading to incorrect predictions; and difficulty in detecting subtle tampering boundaries in inpainted images. In response, we propose a new paradigm that treats IIL as a conditional mask generation task utilizing diffusion models. Our method, InpDiffusion, utilizes the denoising process enhanced by the integration of image semantic conditions to progressively refine predictions. During denoising, we employ edge conditions and introduce a novel edge supervision strategy to enhance the model's perception of edge details in inpainted objects. Balancing the diffusion model's stochastic sampling with edge supervision of tampered image regions mitigates the risk of incorrect predictions from overconfidence and prevents the loss of subtle boundaries that can result from overly stochastic processes. Furthermore, we propose an innovative Dual-stream Multi-scale Feature Extractor (DMFE) for extracting multi-scale features, enhancing feature representation by considering both semantic and edge conditions of the inpainted images. Extensive experiments across challenging datasets demonstrate that the InpDiffusion significantly outperforms existing state-of-the-art methods in IIL tasks, while also showcasing excellent generalization capabilities and robustness.