Fake Speech Wild: Detecting Deepfake Speech on Social Media Platform

TL;DR

This paper introduces the Fake Speech Wild dataset and benchmarks self-supervised learning methods to improve deepfake speech detection on social media, achieving a low error rate in real-world scenarios.

Contribution

It presents the FSW dataset for real-world deepfake speech detection and evaluates SSL-based countermeasures with data augmentation to enhance robustness.

Findings

Achieved an average EER of 3.54% in real-world detection

Demonstrated the effectiveness of data augmentation strategies

Established a benchmark for social media deepfake speech detection

Abstract

The rapid advancement of speech generation technology has led to the widespread proliferation of deepfake speech across social media platforms. While deepfake audio countermeasures (CMs) achieve promising results on public datasets, their performance degrades significantly in cross-domain scenarios. To advance CMs for real-world deepfake detection, we first propose the Fake Speech Wild (FSW) dataset, which includes 254 hours of real and deepfake audio from four different media platforms, focusing on social media. As CMs, we establish a benchmark using public datasets and advanced selfsupervised learning (SSL)-based CMs to evaluate current CMs in real-world scenarios. We also assess the effectiveness of data augmentation strategies in enhancing CM robustness for detecting deepfake speech on social media. Finally, by augmenting public datasets and incorporating the FSW training set, we significantly advanced real-world deepfake audio detection performance, achieving an average equal error rate (EER) of 3.54% across all evaluation sets.