Uncovering Critical Features for Deepfake Detection through the Lottery Ticket Hypothesis

TL;DR

This paper applies the Lottery Ticket Hypothesis to deepfake detection, identifying sparse subnetworks that maintain high accuracy, thereby enabling more efficient and interpretable models suitable for resource-limited environments.

Contribution

It demonstrates that deepfake detection networks contain winning tickets that preserve performance at high sparsity, and introduces an LTH-based pruning method for efficient model deployment.

Findings

Deepfake detection networks have winning tickets at high sparsity levels.

Pruned models retain over 90% of baseline accuracy with 80% sparsity.

LTH-based pruning outperforms one-shot pruning methods.

Abstract

Recent advances in deepfake technology have created increasingly convincing synthetic media that poses significant challenges to information integrity and social trust. While current detection methods show promise, their underlying mechanisms remain poorly understood, and the large sizes of their models make them challenging to deploy in resource-limited environments. This study investigates the application of the Lottery Ticket Hypothesis (LTH) to deepfake detection, aiming to identify the key features crucial for recognizing deepfakes. We examine how neural networks can be efficiently pruned while maintaining high detection accuracy. Through extensive experiments with MesoNet, CNN-5, and ResNet-18 architectures on the OpenForensic and FaceForensics++ datasets, we find that deepfake detection networks contain winning tickets, i.e., subnetworks, that preserve performance even at substantial sparsity levels. Our results indicate that MesoNet retains 56.2% accuracy at 80% sparsity on the OpenForensic dataset, with only 3,000 parameters, which is about 90% of its baseline accuracy (62.6%). The results also show that our proposed LTH-based iterative magnitude pruning approach consistently outperforms one-shot pruning methods. Using Grad-CAM visualization, we analyze how pruned networks maintain their focus on critical facial regions for deepfake detection. Additionally, we demonstrate the transferability of winning tickets across datasets, suggesting potential for efficient, deployable deepfake detection systems.