IndieFake Dataset: A Benchmark Dataset for Audio Deepfake Detection

TL;DR

The IndieFake Dataset provides a diverse, balanced audio deepfake benchmark with South-Asian speakers, addressing limitations of existing datasets and improving detection model robustness across ethnicities.

Contribution

This work introduces the IndieFake Dataset, featuring diverse Indian speakers, balanced data, and speaker-level info, filling gaps in existing audio deepfake datasets.

Findings

IFD outperforms existing datasets in deepfake detection tasks.

Models trained on IFD generalize better across diverse accents.

IFD is more challenging than benchmark ITW dataset.

Abstract

Advancements in audio deepfake technology offers benefits like AI assistants, better accessibility for speech impairments, and enhanced entertainment. However, it also poses significant risks to security, privacy, and trust in digital communications. Detecting and mitigating these threats requires comprehensive datasets. Existing datasets lack diverse ethnic accents, making them inadequate for many real-world scenarios. Consequently, models trained on these datasets struggle to detect audio deepfakes in diverse linguistic and cultural contexts such as in South-Asian countries. Ironically, there is a stark lack of South-Asian speaker samples in the existing datasets despite constituting a quarter of the worlds population. This work introduces the IndieFake Dataset (IFD), featuring 27.17 hours of bonafide and deepfake audio from 50 English speaking Indian speakers. IFD offers balanced data distribution and includes speaker-level characterization, absent in datasets like ASVspoof21 (DF). We evaluated various baselines on IFD against existing ASVspoof21 (DF) and In-The-Wild (ITW) datasets. IFD outperforms ASVspoof21 (DF) and proves to be more challenging compared to benchmark ITW dataset. The complete dataset, along with documentation and sample reference clips, is publicly accessible for research use on project website.