Not All Deepfakes Are Created Equal: Triaging Audio Forgeries for Robust Deepfake Singer Identification

TL;DR

This paper presents a two-stage system to detect high-quality singing voice deepfakes by filtering out low-quality forgeries and then identifying singers in the remaining audio, improving robustness against sophisticated forgeries.

Contribution

It introduces a novel two-stage pipeline combining quality filtering and singer identification, enhancing deepfake detection accuracy for singing voices.

Findings

Outperforms existing baselines on synthetic and authentic audio

Effectively filters out low-quality deepfakes before identification

Improves robustness in singer identification tasks

Abstract

The proliferation of highly realistic singing voice deepfakes presents a significant challenge to protecting artist likeness and content authenticity. Automatic singer identification in vocal deepfakes is a promising avenue for artists and rights holders to defend against unauthorized use of their voice, but remains an open research problem. Based on the premise that the most harmful deepfakes are those of the highest quality, we introduce a two-stage pipeline to identify a singer's vocal likeness. It first employs a discriminator model to filter out low-quality forgeries that fail to accurately reproduce vocal likeness. A subsequent model, trained exclusively on authentic recordings, identifies the singer in the remaining high-quality deepfakes and authentic audio. Experiments show that this system consistently outperforms existing baselines on both authentic and synthetic content.