A General Model for Deepfake Speech Detection: Diverse Bonafide Resources or Diverse AI-Based Generators

TL;DR

This paper investigates how diverse bonafide resources and AI generators influence deepfake speech detection, proposing a balanced dataset and demonstrating that resource balance is key for generalization.

Contribution

It introduces a dataset balancing bonafide resources and AI generators, and shows that this balance improves the generality of deepfake speech detection models.

Findings

Balanced datasets enhance cross-dataset performance.

Resource diversity significantly impacts detection threshold stability.

Cross-dataset evaluation confirms the importance of resource balance.

Abstract

In this paper, we analyze two main factors of Bonafide Resource (BR) or AI-based Generator (AG) which affect the performance and the generality of a Deepfake Speech Detection (DSD) model. To this end, we first propose a deep-learning based model, referred to as the baseline. Then, we conducted experiments on the baseline by which we indicate how Bonafide Resource (BR) and AI-based Generator (AG) factors affect the threshold score used to detect fake or bonafide input audio in the inference process. Given the experimental results, a dataset, which re-uses public Deepfake Speech Detection (DSD) datasets and shows a balance between Bonafide Resource (BR) or AI-based Generator (AG), is proposed. We then train various deep-learning based models on the proposed dataset and conduct cross-dataset evaluation on different benchmark datasets. The cross-dataset evaluation results prove that the balance of Bonafide Resources (BR) and AI-based Generators (AG) is the key factor to train and achieve a general Deepfake Speech Detection (DSD) model.