Invisible Watermarking for Audio Generation Diffusion Models

TL;DR

This paper introduces an invisible watermarking technique for audio diffusion models trained on mel-spectrograms, enabling model verification and ownership protection without compromising audio generation quality.

Contribution

It is the first to apply watermarking to audio diffusion models, providing a novel method for safeguarding model integrity and copyright in audio data generation.

Findings

Watermark triggers effectively protect against unauthorized modifications.

The watermarking method maintains high audio generation quality.

The approach enables reliable model ownership verification.

Abstract

Diffusion models have gained prominence in the image domain for their capabilities in data generation and transformation, achieving state-of-the-art performance in various tasks in both image and audio domains. In the rapidly evolving field of audio-based machine learning, safeguarding model integrity and establishing data copyright are of paramount importance. This paper presents the first watermarking technique applied to audio diffusion models trained on mel-spectrograms. This offers a novel approach to the aforementioned challenges. Our model excels not only in benign audio generation, but also incorporates an invisible watermarking trigger mechanism for model verification. This watermark trigger serves as a protective layer, enabling the identification of model ownership and ensuring its integrity. Through extensive experiments, we demonstrate that invisible watermark triggers can effectively protect against unauthorized modifications while maintaining high utility in benign audio generation tasks.