DinoLizer: Learning from the Best for Generative Inpainting Localization

TL;DR

DinoLizer leverages a DINOv2-based model with a linear head and sliding-window approach to accurately localize manipulated regions in generative inpainting, outperforming existing methods and showing robustness to common post-processing techniques.

Contribution

The paper introduces DinoLizer, a novel DINOv2-based framework with a linear classification head and sliding-window strategy for improved manipulation localization in generative inpainting.

Findings

DinoLizer surpasses state-of-the-art in manipulation detection accuracy.

It remains robust under resizing, noise, and compression.

Achieves 12% higher IoU than previous models.

Abstract

We introduce DinoLizer, a DINOv2-based model for localizing manipulated regions in generative inpainting. Our method builds on a DINOv2 model pretrained to detect synthetic images on the B-Free dataset. We add a linear classification head on top of the Vision Transformer's patch embeddings to predict manipulations at a $14\times 14$ patch resolution. The head is trained to focus on semantically altered regions, treating non-semantic edits as part of the original content. Because the ViT accepts only fixed-size inputs, we use a sliding-window strategy to aggregate predictions over larger images; the resulting heatmaps are post-processed to refine the estimated binary manipulation masks. Empirical results show that DinoLizer surpasses state-of-the-art local manipulation detectors on a range of inpainting datasets derived from different generative models. It remains robust to common post-processing operations such as resizing, noise addition, and JPEG (double) compression. On average, DinoLizer achieves a 12\% higher Intersection-over-Union (IoU) than the next best model, with even greater gains after post-processing. Our experiments with off-the-shelf DINOv2 demonstrate the strong representational power of Vision Transformers for this task. Finally, extensive ablation studies comparing DINOv2 and its successor, DINOv3, in deepfake localization confirm DinoLizer's superiority. The code will be publicly available upon acceptance of the paper.