On the Exploitation of Deepfake Model Recognition

TL;DR

This paper presents a robust method for recognizing specific GAN models used to generate deepfake images by analyzing latent space fingerprints, achieving over 96% classification accuracy.

Contribution

It introduces a novel pipeline for deepfake model recognition that leverages latent space analysis and a dedicated metric, improving identification accuracy for similar models.

Findings

Achieved over 96% accuracy in model classification.

Developed a metric with more than 94% accuracy on unseen models.

Demonstrated the feasibility of fingerprinting deepfake models.

Abstract

Despite recent advances in Generative Adversarial Networks (GANs), with special focus to the Deepfake phenomenon there is no a clear understanding neither in terms of explainability nor of recognition of the involved models. In particular, the recognition of a specific GAN model that generated the deepfake image compared to many other possible models created by the same generative architecture (e.g. StyleGAN) is a task not yet completely addressed in the state-of-the-art. In this work, a robust processing pipeline to evaluate the possibility to point-out analytic fingerprints for Deepfake model recognition is presented. After exploiting the latent space of 50 slightly different models through an in-depth analysis on the generated images, a proper encoder was trained to discriminate among these models obtaining a classification accuracy of over 96%. Once demonstrated the possibility to discriminate extremely similar images, a dedicated metric exploiting the insights discovered in the latent space was introduced. By achieving a final accuracy of more than 94% for the Model Recognition task on images generated by models not employed in the training phase, this study takes an important step in countering the Deepfake phenomenon introducing a sort of signature in some sense similar to those employed in the multimedia forensics field (e.g. for camera source identification task, image ballistics task, etc).