Robust Distortion-Free Watermark for Autoregressive Audio Generation Models

TL;DR

This paper introduces Aligned-IS, a new distortion-free watermarking method for autoregressive audio models that enhances security and detectability without compromising audio quality.

Contribution

The paper presents Aligned-IS, a clustering-based watermarking technique specifically designed to overcome retokenization mismatch in autoregressive audio models, improving security measures.

Findings

Aligned-IS outperforms existing methods in detectability.

It maintains high audio quality post-watermarking.

It sets a new benchmark for secure audio generation.

Abstract

The rapid advancement of next-token-prediction models has led to widespread adoption across modalities, enabling the creation of realistic synthetic media. In the audio domain, while autoregressive speech models have propelled conversational interactions forward, the potential for misuse, such as impersonation in phishing schemes or crafting misleading speech recordings, has also increased. Security measures such as watermarking have thus become essential to ensuring the authenticity of digital media. Traditional statistical watermarking methods used for autoregressive language models face challenges when applied to autoregressive audio models, due to the inevitable ``retokenization mismatch'' - the discrepancy between original and retokenized discrete audio token sequences. To address this, we introduce Aligned-IS, a novel, distortion-free watermark, specifically crafted for audio generation models. This technique utilizes a clustering approach that treats tokens within the same cluster equivalently, effectively countering the retokenization mismatch issue. Our comprehensive testing on prevalent audio generation platforms demonstrates that Aligned-IS not only preserves the quality of generated audio but also significantly improves the watermark detectability compared to the state-of-the-art distortion-free watermarking adaptations, establishing a new benchmark in secure audio technology applications.