Collecting, Curating, and Annotating Good Quality Speech deepfake dataset for Famous Figures: Process and Challenges

TL;DR

This paper details a comprehensive process for creating high-quality speech datasets of public figures, addressing challenges in collecting, curating, and synthesizing authentic and synthetic speech for deepfake detection and research.

Contribution

It introduces an automated pipeline for high-quality speech collection and a systematic approach for synthesizing speech, improving dataset quality for public figure voices.

Findings

Achieved a NISQA-TTS naturalness score of 3.69

Synthetic speech from the dataset was misclassified as real 61.9% of the time

Developed a methodology applicable to political figure speech datasets

Abstract

Recent advances in speech synthesis have introduced unprecedented challenges in maintaining voice authenticity, particularly concerning public figures who are frequent targets of impersonation attacks. This paper presents a comprehensive methodology for collecting, curating, and generating synthetic speech data for political figures and a detailed analysis of challenges encountered. We introduce a systematic approach incorporating an automated pipeline for collecting high-quality bonafide speech samples, featuring transcription-based segmentation that significantly improves synthetic speech quality. We experimented with various synthesis approaches; from single-speaker to zero-shot synthesis, and documented the evolution of our methodology. The resulting dataset comprises bonafide and synthetic speech samples from ten public figures, demonstrating superior quality with a NISQA-TTS naturalness score of 3.69 and the highest human misclassification rate of 61.9\%.