DepthFake: a depth-based strategy for detecting Deepfake videos

TL;DR

DepthFake introduces a novel depth-map based method that enhances deepfake detection accuracy by integrating monocular depth estimation with RGB analysis, significantly improving robustness over traditional RGB-only approaches.

Contribution

This paper presents DepthFake, a new approach combining depth-maps with RGB images to improve deepfake detection, leveraging monocular depth estimation techniques.

Findings

DepthFake improves detection accuracy by an average of 3.20%.

Depth information enhances robustness against various deepfake attacks.

The approach achieves up to 11.7% improvement on certain attacks.

Abstract

Fake content has grown at an incredible rate over the past few years. The spread of social media and online platforms makes their dissemination on a large scale increasingly accessible by malicious actors. In parallel, due to the growing diffusion of fake image generation methods, many Deep Learning-based detection techniques have been proposed. Most of those methods rely on extracting salient features from RGB images to detect through a binary classifier if the image is fake or real. In this paper, we proposed DepthFake, a study on how to improve classical RGB-based approaches with depth-maps. The depth information is extracted from RGB images with recent monocular depth estimation techniques. Here, we demonstrate the effective contribution of depth-maps to the deepfake detection task on robust pre-trained architectures. The proposed RGBD approach is in fact able to achieve an average improvement of 3.20% and up to 11.7% for some deepfake attacks with respect to standard RGB architectures over the FaceForensic++ dataset.