Speech Watermarking with Discrete Intermediate Representations

TL;DR

DiscreteWM is a novel speech watermarking framework that embeds watermarks into discrete intermediate representations of speech, achieving high robustness and imperceptibility, and capable of encoding up to 150 bits per second.

Contribution

It introduces a discrete latent space watermarking method using vector-quantized autoencoders and a token manipulation strategy for imperceptibility.

Findings

Achieves state-of-the-art robustness and imperceptibility

Can encode 1 to 150 bits of watermark per second

Effective for voice cloning detection and information hiding

Abstract

Speech watermarking techniques can proactively mitigate the potential harmful consequences of instant voice cloning techniques. These techniques involve the insertion of signals into speech that are imperceptible to humans but can be detected by algorithms. Previous approaches typically embed watermark messages into continuous space. However, intuitively, embedding watermark information into robust discrete latent space can significantly improve the robustness of watermarking systems. In this paper, we propose DiscreteWM, a novel speech watermarking framework that injects watermarks into the discrete intermediate representations of speech. Specifically, we map speech into discrete latent space with a vector-quantized autoencoder and inject watermarks by changing the modular arithmetic relation of discrete IDs. To ensure the imperceptibility of watermarks, we also propose a manipulator model to select the candidate tokens for watermark embedding. Experimental results demonstrate that our framework achieves state-of-the-art performance in robustness and imperceptibility, simultaneously. Moreover, our flexible frame-wise approach can serve as an efficient solution for both voice cloning detection and information hiding. Additionally, DiscreteWM can encode 1 to 150 bits of watermark information within a 1-second speech clip, indicating its encoding capacity. Audio samples are available at https://DiscreteWM.github.io/discrete_wm.