Where Do Deep Fakes Look? Synthetic Face Detection via Gaze Tracking

TL;DR

This paper introduces a novel deep fake detection method based on gaze and eye features, leveraging biological signals to distinguish real videos from fakes with high accuracy across multiple datasets.

Contribution

It proposes specific gaze and eye features as authenticity signatures and integrates them into a neural network for effective deep fake detection in diverse scenarios.

Findings

Achieves over 92% accuracy on FaceForensics++

Outperforms existing detectors without complex architectures

Effective across multiple deep fake datasets

Abstract

Following the recent initiatives for the democratization of AI, deep fake generators have become increasingly popular and accessible, causing dystopian scenarios towards social erosion of trust. A particular domain, such as biological signals, attracted attention towards detection methods that are capable of exploiting authenticity signatures in real videos that are not yet faked by generative approaches. In this paper, we first propose several prominent eye and gaze features that deep fakes exhibit differently. Second, we compile those features into signatures and analyze and compare those of real and fake videos, formulating geometric, visual, metric, temporal, and spectral variations. Third, we generalize this formulation to the deep fake detection problem by a deep neural network, to classify any video in the wild as fake or real. We evaluate our approach on several deep fake datasets, achieving 92.48% accuracy on FaceForensics++, 80.0% on Deep Fakes (in the wild), 88.35% on CelebDF, and 99.27% on DeeperForensics datasets. Our approach outperforms most deep and biological fake detectors with complex network architectures without the proposed gaze signatures. We conduct ablation studies involving different features, architectures, sequence durations, and post-processing artifacts.