DeePen: Penetration Testing for Audio Deepfake Detection
Nicolas M\"uller, Piotr Kawa, Adriana Stan, Thien-Phuc Doan, Souhwan Jung, Wei Herng Choong, Philip Sperl, Konstantin B\"ottinger

TL;DR
This paper introduces DeePen, a systematic method for testing the robustness of deepfake audio detectors using signal processing attacks, revealing widespread vulnerabilities in current systems.
Contribution
DeePen is a novel, model-agnostic penetration testing approach that uncovers vulnerabilities in deepfake audio detection systems without prior model access.
Findings
All tested systems are vulnerable to simple signal manipulations.
Some attacks can be mitigated by retraining, others remain effective.
Real-world and academic models both exhibit weaknesses.
Abstract
Deepfakes - manipulated or forged audio and video media - pose significant security risks to individuals, organizations, and society at large. To address these challenges, machine learning-based classifiers are commonly employed to detect deepfake content. In this paper, we assess the robustness of such classifiers through a systematic penetration testing methodology, which we introduce as DeePen. Our approach operates without prior knowledge of or access to the target deepfake detection models. Instead, it leverages a set of carefully selected signal processing modifications - referred to as attacks - to evaluate model vulnerabilities. Using DeePen, we analyze both real-world production systems and publicly available academic model checkpoints, demonstrating that all tested systems exhibit weaknesses and can be reliably deceived by simple manipulations such as time-stretching or echo…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
