Improving Generalization in Deepfake Detection with Face Foundation Models and Metric Learning

TL;DR

This paper introduces a deepfake detection framework that leverages face foundation models and metric learning techniques to improve generalization across diverse and challenging real-world media scenarios.

Contribution

It combines face foundation models with triplet loss and attribution-based supervision to enhance deepfake detection generalization beyond training distributions.

Findings

Improved detection accuracy on diverse benchmarks.

Enhanced generalization to unseen deepfake types.

Effective use of face foundation models and metric learning.

Abstract

The increasing realism and accessibility of deepfakes have raised critical concerns about media authenticity and information integrity. Despite recent advances, deepfake detection models often struggle to generalize beyond their training distributions, particularly when applied to media content found in the wild. In this work, we present a robust video deepfake detection framework with strong generalization that takes advantage of the rich facial representations learned by face foundation models. Our method is built on top of FSFM, a self-supervised model trained on real face data, and is further fine-tuned using an ensemble of deepfake datasets spanning both face-swapping and face-reenactment manipulations. To enhance discriminative power, we incorporate triplet loss variants during training, guiding the model to produce more separable embeddings between real and fake samples. Additionally, we explore attribution-based supervision schemes, where deepfakes are categorized by manipulation type or source dataset, to assess their impact on generalization. Extensive experiments across diverse evaluation benchmarks demonstrate the effectiveness of our approach, especially in challenging real-world scenarios.