Evaluating Generalization and Robustness in Russian Anti-Spoofing: The RuASD Initiative

TL;DR

The RuASD dataset provides a comprehensive benchmark for evaluating Russian speech anti-spoofing methods, focusing on generalization and robustness across diverse conditions and perturbations.

Contribution

It introduces a reproducible Russian speech anti-spoofing benchmark with diverse data, realistic distortions, and extensive evaluation of various anti-spoofing models.

Findings

Benchmark results show varying robustness of models under simulated distortions.

The dataset enables systematic evaluation of anti-spoofing methods in realistic scenarios.

Reference results highlight the importance of robustness in deployment.

Abstract

RuASD (Russian AntiSpoofing Dataset) is a dedicated, reproducible benchmark for Russian-language speech anti-spoofing designed to evaluate both in-domain discrimination and robustness to deployment-style distribution shifts. It combines a large spoof subset synthesized using 37 modern Russian-capable TTS and voice-cloning systems with a bona fide subset curated from multiple heterogeneous open Russian speech corpora, enabling systematic evaluation across diverse data sources. To emulate typical dissemination and channel effects in a controlled and reproducible manner, RuASD includes configurable simulations of platform and transmission distortions, including room reverberation, additive noise/music, and a range of speech-codec transcodings implemented via a unified processing chain. We benchmark a diverse set of publicly available anti-spoofing countermeasures spanning lightweight supervised architectures, graph-attention models, SSL-based detectors, and large-scale pretrained systems, and report reference results on both clean and simulated conditions to characterize robustness under realistic perturbation pipelines. The dataset is publickly available at \href{https://huggingface.co/datasets/MTUCI/RuASD}{\underline{Hugging Face}} and \href{https://modelscope.cn/datasets/lab260/RuASD}{\underline{ModelScope}}.