Memory-Anchored Multimodal Reasoning for Explainable Video Forensics

TL;DR

FakeHunter is a novel multimodal framework for video fake detection that combines memory-guided retrieval, reasoning loops, and adaptive tool use to improve robustness and explainability in detecting manipulations.

Contribution

It introduces FakeHunter, integrating memory retrieval, reasoning, and adaptive analysis, along with a new benchmark for comprehensive evaluation of multimodal fake detection methods.

Findings

FakeHunter outperforms existing multimodal baselines.

Memory-guided retrieval enhances contextual understanding.

Selective analysis improves detection robustness and explanation quality.

Abstract

We address multimodal deepfake detection requiring both robustness and interpretability by proposing FakeHunter, a unified framework that combines memory guided retrieval, a structured Observation-Thought-Action reasoning loop, and adaptive forensic tool invocation. Visual representations from a Contrastive Language-Image Pretraining (CLIP) model and audio representations from a Contrastive Language-Audio Pretraining (CLAP) model retrieve semantically aligned authentic exemplars from a large scale memory, providing contextual anchors that guide iterative localization and explanation of suspected manipulations. Under low internal confidence the framework selectively triggers fine grained analyses such as spatial region zoom and mel spectrogram inspection to gather discriminative evidence instead of relying on opaque marginal scores. We also release X-AVFake, a comprehensive audio visual forgery benchmark with fine grained annotations of manipulation type, affected region or entity, reasoning category, and explanatory justification, designed to stress contextual grounding and explanation fidelity. Extensive experiments show that FakeHunter surpasses strong multimodal baselines, and ablation studies confirm that both contextual retrieval and selective tool activation are indispensable for improved robustness and explanatory precision.