Pindrop it! Audio and Visual Deepfake Countermeasures for Robust Detection and Fine Grained-Localization

TL;DR

This paper introduces robust audio-visual deepfake detection and localization methods that excel in identifying and pinpointing synthetic manipulations in videos, addressing challenges posed by subtle, localized alterations.

Contribution

It presents novel solutions for deepfake classification and localization, achieving top performance in a major challenge and advancing detection capabilities.

Findings

Achieved best performance in temporal localization task.

Ranked top four in deepfake classification.

Effective detection of subtle, localized manipulations.

Abstract

The field of visual and audio generation is burgeoning with new state-of-the-art methods. This rapid proliferation of new techniques underscores the need for robust solutions for detecting synthetic content in videos. In particular, when fine-grained alterations via localized manipulations are performed in visual, audio, or both domains, these subtle modifications add challenges to the detection algorithms. This paper presents solutions for the problems of deepfake video classification and localization. The methods were submitted to the ACM 1M Deepfakes Detection Challenge, achieving the best performance in the temporal localization task and a top four ranking in the classification task for the TestA split of the evaluation dataset.