Use of a Capsule Network to Detect Fake Images and Videos

TL;DR

This paper introduces a capsule network-based method for detecting various types of fake images and videos, demonstrating its effectiveness and providing the first theoretical analysis of capsule networks in digital forensics.

Contribution

It presents a novel application of capsule networks for multimedia forensics, with a detailed theoretical explanation and visualization of their effectiveness in detecting deepfake attacks.

Findings

Capsule network achieves high detection accuracy with fewer parameters.

Effective against multiple attack types including printed, replayed, and deepfake videos.

Provides the first theoretical analysis of capsule networks in digital forensics.

Abstract

The revolution in computer hardware, especially in graphics processing units and tensor processing units, has enabled significant advances in computer graphics and artificial intelligence algorithms. In addition to their many beneficial applications in daily life and business, computer-generated/manipulated images and videos can be used for malicious purposes that violate security systems, privacy, and social trust. The deepfake phenomenon and its variations enable a normal user to use his or her personal computer to easily create fake videos of anybody from a short real online video. Several countermeasures have been introduced to deal with attacks using such videos. However, most of them are targeted at certain domains and are ineffective when applied to other domains or new attacks. In this paper, we introduce a capsule network that can detect various kinds of attacks, from presentation attacks using printed images and replayed videos to attacks using fake videos created using deep learning. It uses many fewer parameters than traditional convolutional neural networks with similar performance. Moreover, we explain, for the first time ever in the literature, the theory behind the application of capsule networks to the forensics problem through detailed analysis and visualization.