Double Entendre: Robust Audio-Based AI-Generated Lyrics Detection via Multi-View Fusion

TL;DR

This paper introduces DE-detect, a multimodal, robust audio-lyrics fusion method for detecting AI-generated music, overcoming limitations of existing single-modality detectors in real-world scenarios.

Contribution

The paper presents a novel multimodal late-fusion pipeline that combines transcribed lyrics and speech features for robust AI-generated music detection, improving over existing methods.

Findings

DE-detect outperforms existing lyrics-based detectors.

The method is more robust to audio perturbations.

It demonstrates practical applicability in real-world scenarios.

Abstract

The rapid advancement of AI-based music generation tools is revolutionizing the music industry but also posing challenges to artists, copyright holders, and providers alike. This necessitates reliable methods for detecting such AI-generated content. However, existing detectors, relying on either audio or lyrics, face key practical limitations: audio-based detectors fail to generalize to new or unseen generators and are vulnerable to audio perturbations; lyrics-based methods require cleanly formatted and accurate lyrics, unavailable in practice. To overcome these limitations, we propose a novel, practically grounded approach: a multimodal, modular late-fusion pipeline that combines automatically transcribed sung lyrics and speech features capturing lyrics-related information within the audio. By relying on lyrical aspects directly from audio, our method enhances robustness, mitigates susceptibility to low-level artifacts, and enables practical applicability. Experiments show that our method, DE-detect, outperforms existing lyrics-based detectors while also being more robust to audio perturbations. Thus, it offers an effective, robust solution for detecting AI-generated music in real-world scenarios. Our code is available at https://github.com/deezer/robust-AI-lyrics-detection.