Echoes: A semantically-aligned music deepfake detection dataset

TL;DR

Echoes is a challenging new dataset for music deepfake detection that emphasizes semantic alignment and diversity, leading to improved generalization of detection models across different datasets.

Contribution

The paper introduces Echoes, a large, diverse, and semantically-aligned music deepfake dataset designed to enhance training and benchmarking of detection methods.

Findings

Echoes is the most challenging in-domain dataset.

Detectors trained on existing datasets perform poorly on Echoes.

Training on Echoes improves generalization to other datasets.

Abstract

We introduce Echoes, a new dataset for music deepfake detection designed for training and benchmarking detectors under realistic and provider-diverse conditions. Echoes comprises 3,577 tracks (110 hours of audio) spanning multiple genres (pop, rock, electronic), and includes content generated by ten popular AI music generation systems. To prevent shortcut learning and promote robust generalization, the dataset is deliberately constructed to be challenging, enforcing semantic-level alignment between spoofed audio and bona fide references. This alignment is achieved by conditioning generated audio samples directly on bona-fide waveforms or song descriptors. We evaluate Echoes in a cross-dataset setting against three existing AI-generated music datasets using state-of-the-art Wav2Vec2 XLS-R 2B representations. Results show that (i) Echoes is the hardest in-domain dataset; (ii) detectors trained on existing datasets transfer poorly to Echoes; (iii) training on Echoes yields the strongest generalization performance. These findings suggest that provider diversity and semantic alignment help learn more transferable detection cues.