A Hybrid Deep Learning and Forensic Approach for Robust Deepfake Detection

TL;DR

This paper introduces a hybrid deep learning and forensic method for deepfake detection that combines multiple features, outperforming existing approaches in accuracy, robustness, and explainability across benchmark datasets.

Contribution

It proposes a novel hybrid framework integrating forensic features with deep learning models, enhancing detection accuracy, robustness, and interpretability against evolving deepfake techniques.

Findings

Achieved high F1-scores of 0.96, 0.82, and 0.77 on benchmark datasets.

Demonstrated robustness under compression, adversarial attacks, and unseen manipulations.

Showed improved explainability with 82% overlap between heatmaps and manipulated regions.

Abstract

The rapid evolution of generative adversarial networks (GANs) and diffusion models has made synthetic media increasingly realistic, raising societal concerns around misinformation, identity fraud, and digital trust. Existing deepfake detection methods either rely on deep learning, which suffers from poor generalization and vulnerability to distortions, or forensic analysis, which is interpretable but limited against new manipulation techniques. This study proposes a hybrid framework that fuses forensic features, including noise residuals, JPEG compression traces, and frequency-domain descriptors, with deep learning representations from convolutional neural networks (CNNs) and vision transformers (ViTs). Evaluated on benchmark datasets (FaceForensics++, Celeb-DF v2, DFDC), the proposed model consistently outperformed single-method baselines and demonstrated superior performance compared to existing state-of-the-art hybrid approaches, achieving F1-scores of 0.96, 0.82, and 0.77, respectively. Robustness tests demonstrated stable performance under compression (F1 = 0.87 at QF = 50), adversarial perturbations (AUC = 0.84), and unseen manipulations (F1 = 0.79). Importantly, explainability analysis showed that Grad-CAM and forensic heatmaps overlapped with ground-truth manipulated regions in 82 percent of cases, enhancing transparency and user trust. These findings confirm that hybrid approaches provide a balanced solution, combining the adaptability of deep models with the interpretability of forensic cues, to develop resilient and trustworthy deepfake detection systems.