GEM-TFL: Bridging Weak and Full Supervision for Forgery Localization through EM-Guided Decomposition and Temporal Refinement

TL;DR

GEM-TFL introduces a novel framework that improves weakly supervised temporal forgery localization by leveraging EM-based attribute reformulation, temporal refinement, and graph-based proposal modeling, achieving results close to fully supervised methods.

Contribution

The paper presents GEM-TFL, a two-phase framework that bridges the supervision gap in temporal forgery localization using EM optimization, temporal consistency refinement, and graph-based proposal modeling.

Findings

Achieves more accurate forgery localization than previous weakly supervised methods.

Narrowed the performance gap between weakly and fully supervised approaches.

Demonstrates robustness and effectiveness on benchmark datasets.

Abstract

Temporal Forgery Localization (TFL) aims to precisely identify manipulated segments within videos or audio streams, providing interpretable evidence for multimedia forensics and security. While most existing TFL methods rely on dense frame-level labels in a fully supervised manner, Weakly Supervised TFL (WS-TFL) reduces labeling cost by learning only from binary video-level labels. However, current WS-TFL approaches suffer from mismatched training and inference objectives, limited supervision from binary labels, gradient blockage caused by non-differentiable top-k aggregation, and the absence of explicit modeling of inter-proposal relationships. To address these issues, we propose GEM-TFL (Graph-based EM-powered Temporal Forgery Localization), a two-phase classification-regression framework that effectively bridges the supervision gap between training and inference. Built upon this foundation, (1) we enhance weak supervision by reformulating binary labels into multi-dimensional latent attributes through an EM-based optimization process; (2) we introduce a training-free temporal consistency refinement that realigns frame-level predictions for smoother temporal dynamics; and (3) we design a graph-based proposal refinement module that models temporal-semantic relationships among proposals for globally consistent confidence estimation. Extensive experiments on benchmark datasets demonstrate that GEM-TFL achieves more accurate and robust temporal forgery localization, substantially narrowing the gap with fully supervised methods.