The Deepfakes We Missed: We Built Detectors for a Threat That Didn't Arrive

TL;DR

This paper argues that deepfake detection research has focused on outdated threats, while recent harms like NCII and scams are different, requiring a realignment of research priorities.

Contribution

It provides empirical analysis of recent deepfake harms, highlights the misalignment in research focus, and proposes new technical research directions.

Findings

Large-scale deepfake threats did not materialize as predicted.

Recent harms include NCII, scam calls, and emotional manipulation.

Research efforts remain concentrated on outdated threat models.

Abstract

Nearly a decade of Machine Learning (ML) research on deepfake detection has been organized around a threat model inherited from 2017--2019, revolving around face-swap and talking-head manipulation of public figures, motivated by concerns about large-scale misinformation and video-evidence fraud. This position paper argues that the threat the field prepared for did not arrive, and the threats that did arrive are substantially different. An accounting of deepfake incidents in 2022--2026 shows that the dominant observed harms are peer-generated Non-Consensual Intimate Imagery (NCII), voice-clone scam calls targeting families and finance workers, and emotional-manipulation fraud. The predicted large-scale public-figure deepfake catastrophe did not materialize during the 2024 global information environment despite extensive preparation. Meanwhile, research effort, benchmarks, and detection methods remain concentrated on the inherited threat model. The central claim of this paper is that this misalignment is now the dominant bottleneck on real-world deepfake defense, not model capability. We argue the ML research community should substantially rebalance its research agenda toward the harm categories that are actually growing. We support this position with empirical accounting of research effort and harm distribution, identify the structural reasons the misalignment persists, and outline three concrete technical research agendas for the under-defended harm categories.