A Hybrid CNN-LSTM model for Video Deepfake Detection by Leveraging Optical Flow Features

TL;DR

This paper proposes a hybrid CNN-LSTM model utilizing optical flow features to improve deepfake video detection by capturing temporal dynamics, achieving high accuracy with limited sample sizes on multiple datasets.

Contribution

The study introduces a novel hybrid CNN-LSTM approach that leverages optical flow for temporal feature extraction, enhancing deepfake detection performance over traditional spatial-only methods.

Findings

Achieved 66.26% accuracy on DFDC dataset.

Achieved 91.21% accuracy on FF++ dataset.

Achieved 79.49% accuracy on Celeb-DF dataset.

Abstract

Deepfakes are the synthesized digital media in order to create ultra-realistic fake videos to trick the spectator. Deep generative algorithms, such as, Generative Adversarial Networks(GAN) are widely used to accomplish such tasks. This approach synthesizes pseudo-realistic contents that are very difficult to distinguish by traditional detection methods. In most cases, Convolutional Neural Network(CNN) based discriminators are being used for detecting such synthesized media. However, it emphasise primarily on the spatial attributes of individual video frames, thereby fail to learn the temporal information from their inter-frame relations. In this paper, we leveraged an optical flow based feature extraction approach to extract the temporal features, which are then fed to a hybrid model for classification. This hybrid model is based on the combination of CNN and recurrent neural network (RNN) architectures. The hybrid model provides effective performance on open source data-sets such as, DFDC, FF++ and Celeb-DF. This proposed method shows an accuracy of 66.26%, 91.21% and 79.49% in DFDC, FF++, and Celeb-DF respectively with a very reduced No of sample size of approx 100 samples(frames). This promises early detection of fake contents compared to existing modalities.