Loupe: A Generalizable and Adaptive Framework for Image Forgery Detection

TL;DR

Loupe is a lightweight, adaptive framework that jointly detects and localizes image forgeries, demonstrating state-of-the-art performance and robustness against distribution shifts in deepfake detection tasks.

Contribution

Introduces Loupe, a novel framework combining patch-aware classification and segmentation with test-time adaptation for improved forgery detection and localization.

Findings

Achieves first place in IJCAI 2025 Deepfake Detection Challenge

Outperforms existing methods on the DDL dataset

Enhances robustness through pseudo-label-guided test-time adaptation

Abstract

The proliferation of generative models has raised serious concerns about visual content forgery. Existing deepfake detection methods primarily target either image-level classification or pixel-wise localization. While some achieve high accuracy, they often suffer from limited generalization across manipulation types or rely on complex architectures. In this paper, we propose Loupe, a lightweight yet effective framework for joint deepfake detection and localization. Loupe integrates a patch-aware classifier and a segmentation module with conditional queries, allowing simultaneous global authenticity classification and fine-grained mask prediction. To enhance robustness against distribution shifts of test set, Loupe introduces a pseudo-label-guided test-time adaptation mechanism by leveraging patch-level predictions to supervise the segmentation head. Extensive experiments on the DDL dataset demonstrate that Loupe achieves state-of-the-art performance, securing the first place in the IJCAI 2025 Deepfake Detection and Localization Challenge with an overall score of 0.846. Our results validate the effectiveness of the proposed patch-level fusion and conditional query design in improving both classification accuracy and spatial localization under diverse forgery patterns. The code is available at https://github.com/Kamichanw/Loupe.