Detecting Inpainted Video with Frequency Domain Insights

TL;DR

This paper introduces FDIN, a novel neural network that leverages frequency domain analysis to improve the detection of inpainted videos, addressing limitations of previous spatial-temporal methods.

Contribution

The paper presents FDIN, incorporating frequency domain modules like Adaptive Band Selective Response and Fourier-based Attention, achieving superior detection accuracy in video forensics.

Findings

FDIN outperforms existing methods on public datasets.

Achieves state-of-the-art detection accuracy.

Sets new benchmarks in video inpainting detection.

Abstract

Video inpainting enables seamless content removal and replacement within frames, posing ethical and legal risks when misused. To mitigate these risks, detecting manipulated regions in inpainted videos is critical. Previous detection methods often focus solely on the characteristics derived from spatial and temporal dimensions, which limits their effectiveness by overlooking the unique frequency characteristics of different inpainting algorithms. In this paper, we propose the Frequency Domain Insights Network (FDIN), which significantly enhances detection accuracy by incorporating insights from the frequency domain. Our network features an Adaptive Band Selective Response module to discern frequency characteristics specific to various inpainting techniques and a Fast Fourier Convolution-based Attention module for identifying periodic artifacts in inpainted regions. Utilizing 3D ResBlocks for spatiotemporal analysis, FDIN progressively refines detection precision from broad assessments to detailed localization. Experimental evaluations on public datasets demonstrate that FDIN achieves state-of-the-art performance, setting a new benchmark in video inpainting detection.