A SUPERB-Style Benchmark of Self-Supervised Speech Models for Audio Deepfake Detection

TL;DR

This paper introduces Spoof-SUPERB, a benchmark evaluating 20 self-supervised speech models for audio deepfake detection, revealing that discriminative models outperform others and are more robust under acoustic degradations.

Contribution

It systematically evaluates SSL models for audio deepfake detection, establishing a reproducible benchmark and analyzing model robustness and reliability.

Findings

Discriminative models like XLS-R and WavLM Large outperform others.

Large-scale models benefit from multilingual pretraining and speaker-aware objectives.

Discriminative models are more resilient to acoustic degradations.

Abstract

Self-supervised learning (SSL) has transformed speech processing, with benchmarks such as SUPERB establishing fair comparisons across diverse downstream tasks. Despite it's security-critical importance, Audio deepfake detection has remained outside these efforts. In this work, we introduce Spoof-SUPERB, a benchmark for audio deepfake detection that systematically evaluates 20 SSL models spanning generative, discriminative, and spectrogram-based architectures. We evaluated these models on multiple in-domain and out-of-domain datasets. Our results reveal that large-scale discriminative models such as XLS-R, UniSpeech-SAT, and WavLM Large consistently outperform other models, benefiting from multilingual pretraining, speaker-aware objectives, and model scale. We further analyze the robustness of these models under acoustic degradations, showing that generative approaches degrade sharply, while discriminative models remain resilient. This benchmark establishes a reproducible baseline and provides practical insights into which SSL representations are most reliable for securing speech systems against audio deepfakes.