VoiceMark: Zero-Shot Voice Cloning-Resistant Watermarking Approach Leveraging Speaker-Specific Latents

TL;DR

VoiceMark is a novel zero-shot voice cloning-resistant watermarking technique that uses speaker-specific latents to embed watermarks, achieving high detection accuracy even after zero-shot synthesis, thus enhancing audio copyright protection.

Contribution

It introduces the first zero-shot VC-resistant watermarking method leveraging speaker-specific latents, with robustness enhancements like VC-simulated augmentations and VAD-based loss.

Findings

Over 95% watermark detection accuracy after zero-shot VC synthesis

Significantly outperforms existing methods with around 50% accuracy

Effective robustness against distortions in zero-shot voice cloning scenarios

Abstract

Voice cloning (VC)-resistant watermarking is an emerging technique for tracing and preventing unauthorized cloning. Existing methods effectively trace traditional VC models by training them on watermarked audio but fail in zero-shot VC scenarios, where models synthesize audio from an audio prompt without training. To address this, we propose VoiceMark, the first zero-shot VC-resistant watermarking method that leverages speaker-specific latents as the watermark carrier, allowing the watermark to transfer through the zero-shot VC process into the synthesized audio. Additionally, we introduce VC-simulated augmentations and VAD-based loss to enhance robustness against distortions. Experiments on multiple zero-shot VC models demonstrate that VoiceMark achieves over 95% accuracy in watermark detection after zero-shot VC synthesis, significantly outperforming existing methods, which only reach around 50%. See our code and demos at: https://huggingface.co/spaces/haiyunli/VoiceMark