Detecting music deepfakes is easy but actually hard

TL;DR

This paper introduces the first music deepfake detector, demonstrating high accuracy in identifying fake audio, but emphasizes the challenges in robustness, interpretability, and real-world deployment.

Contribution

It presents the initial development of a music deepfake detection method and discusses critical issues for practical application and future research.

Findings

Achieved 99.8% accuracy in detecting music deepfakes.

Identified potential problems like robustness and interpretability in deepfake detectors.

Highlighted the need for further research beyond simple ML models.

Abstract

In the face of a new era of generative models, the detection of artificially generated content has become a matter of utmost importance. The ability to create credible minute-long music deepfakes in a few seconds on user-friendly platforms poses a real threat of fraud on streaming services and unfair competition to human artists. This paper demonstrates the possibility (and surprising ease) of training classifiers on datasets comprising real audio and fake reconstructions, achieving a convincing accuracy of 99.8%. To our knowledge, this marks the first publication of a music deepfake detector, a tool that will help in the regulation of music forgery. Nevertheless, informed by decades of literature on forgery detection in other fields, we stress that a good test score is not the end of the story. We step back from the straightforward ML framework and expose many facets that could be problematic with such a deployed detector: calibration, robustness to audio manipulation, generalisation to unseen models, interpretability and possibility for recourse. This second part acts as a position for future research steps in the field and a caveat to a flourishing market of fake content checkers.