EchoFake: A Replay-Aware Dataset for Practical Speech Deepfake Detection

TL;DR

EchoFake is a new dataset with over 120 hours of diverse replayed and TTS speech, designed to improve the robustness of deepfake detection models in real-world scenarios.

Contribution

The paper introduces EchoFake, a large-scale, realistic dataset for speech deepfake detection, addressing the gap in existing datasets for practical replay attack scenarios.

Findings

Models trained on EchoFake show improved generalization across datasets.

Baseline models achieve lower EERs when trained on EchoFake.

Existing datasets lead to poor performance on replay attacks, highlighting the need for more realistic data.

Abstract

The growing prevalence of speech deepfakes has raised serious concerns, particularly in real-world scenarios such as telephone fraud and identity theft. While many anti-spoofing systems have demonstrated promising performance on lab-generated synthetic speech, they often fail when confronted with physical replay attacks-a common and low-cost form of attack used in practical settings. Our experiments show that models trained on existing datasets exhibit severe performance degradation, with average accuracy dropping to 59.6% when evaluated on replayed audio. To bridge this gap, we present EchoFake, a comprehensive dataset comprising more than 120 hours of audio from over 13,000 speakers, featuring both cutting-edge zero-shot text-to-speech (TTS) speech and physical replay recordings collected under varied devices and real-world environmental settings. Additionally, we evaluate three baseline detection models and show that models trained on EchoFake achieve lower average EERs across datasets, indicating better generalization. By introducing more practical challenges relevant to real-world deployment, EchoFake offers a more realistic foundation for advancing spoofing detection methods.