Anisotropic multiresolution analyses for deepfake detection

TL;DR

This paper introduces anisotropic multiresolution analysis techniques using wavelets and multiwavelets to improve deepfake detection by capturing GAN-specific traces in generated media.

Contribution

It proposes the use of anisotropic transformations, specifically fully separable wavelet transforms and multiwavelets, to enhance deepfake detection over existing isotropic methods.

Findings

Anisotropic features improve detection accuracy.

Fully separable transforms outperform isotropic methods.

State-of-the-art results achieved with proposed approach.

Abstract

Generative Adversarial Networks (GANs) have paved the path towards entirely new media generation capabilities at the forefront of image, video, and audio synthesis. However, they can also be misused and abused to fabricate elaborate lies, capable of stirring up the public debate. The threat posed by GANs has sparked the need to discern between genuine content and fabricated one. Previous studies have tackled this task by using classical machine learning techniques, such as k-nearest neighbours and eigenfaces, which unfortunately did not prove very effective. Subsequent methods have focused on leveraging on frequency decompositions, i.e., discrete cosine transform, wavelets, and wavelet packets, to preprocess the input features for classifiers. However, existing approaches only rely on isotropic transformations. We argue that, since GANs primarily utilize isotropic convolutions to generate their output, they leave clear traces, their fingerprint, in the coefficient distribution on sub-bands extracted by anisotropic transformations. We employ the fully separable wavelet transform and multiwavelets to obtain the anisotropic features to feed to standard CNN classifiers. Lastly, we find the fully separable transform capable of improving the state-of-the-art.