Deep Learning and Synthetic Media

TL;DR

Deep learning has revolutionized audiovisual media creation, enabling highly realistic synthetic media that challenge traditional classifications and raise new ethical and technological questions.

Contribution

The paper analyzes how deep learning-based synthetic media differ from traditional methods and their implications for media taxonomy and future media forms.

Findings

Deepfakes are increasingly realistic and easy to produce.

Deep learning challenges traditional media classifications.

Synthetic media enable new forms of audiovisual expression.

Abstract

Deep learning algorithms are rapidly changing the way in which audiovisual media can be produced. Synthetic audiovisual media generated with deep learning - often subsumed colloquially under the label "deepfakes" - have a number of impressive characteristics; they are increasingly trivial to produce, and can be indistinguishable from real sounds and images recorded with a sensor. Much attention has been dedicated to ethical concerns raised by this technological development. Here, I focus instead on a set of issues related to the notion of synthetic audiovisual media, its place within a broader taxonomy of audiovisual media, and how deep learning techniques differ from more traditional approaches to media synthesis. After reviewing important etiological features of deep learning pipelines for media manipulation and generation, I argue that "deepfakes" and related synthetic media produced with such pipelines do not merely offer incremental improvements over previous methods, but challenge traditional taxonomical distinctions, and pave the way for genuinely novel kinds of audiovisual media.