SCDF: A Speaker Characteristics DeepFake Speech Dataset for Bias Analysis

TL;DR

The paper introduces the SCDF dataset, a large, annotated collection of deepfake speech samples designed to evaluate demographic biases in detection systems, revealing significant disparities across speaker characteristics.

Contribution

It provides a novel, richly annotated dataset for systematic bias analysis in deepfake speech detection, addressing a critical gap in fairness research.

Findings

Detection performance varies significantly across demographic groups.

Speaker characteristics influence deepfake detection accuracy.

The dataset enables comprehensive bias analysis in speech deepfake detection.

Abstract

Despite growing attention to deepfake speech detection, the aspects of bias and fairness remain underexplored in the speech domain. To address this gap, we introduce the Speaker Characteristics Deepfake (SCDF) dataset: a novel, richly annotated resource enabling systematic evaluation of demographic biases in deepfake speech detection. SCDF contains over 237,000 utterances in a balanced representation of both male and female speakers spanning five languages and a wide age range. We evaluate several state-of-the-art detectors and show that speaker characteristics significantly influence detection performance, revealing disparities across sex, language, age, and synthesizer type. These findings highlight the need for bias-aware development and provide a foundation for building non-discriminatory deepfake detection systems aligned with ethical and regulatory standards.