Towards More General Video-based Deepfake Detection through Facial Component Guided Adaptation for Foundation Model

TL;DR

This paper introduces a novel approach for video-based Deepfake detection that leverages foundation models with facial component guidance to improve generalization, efficiency, and robustness against unseen forgeries.

Contribution

It proposes a side-network decoder with facial component guidance that enhances the generalization of foundation models for Deepfake detection in videos.

Findings

Demonstrates improved generalization on challenging Deepfake datasets.

Shows superiority in training data and parameter efficiency.

Achieves increased model robustness against unseen forgeries.

Abstract

Generative models have enabled the creation of highly realistic facial-synthetic images, raising significant concerns due to their potential for misuse. Despite rapid advancements in the field of deepfake detection, developing efficient approaches to leverage foundation models for improved generalizability to unseen forgery samples remains challenging. To address this challenge, we propose a novel side-network-based decoder that extracts spatial and temporal cues using the CLIP image encoder for generalized video-based Deepfake detection. Additionally, we introduce Facial Component Guidance (FCG) to enhance spatial learning generalizability by encouraging the model to focus on key facial regions. By leveraging the generic features of a vision-language foundation model, our approach demonstrates promising generalizability on challenging Deepfake datasets while also exhibiting superiority in training data efficiency, parameter efficiency, and model robustness.