Making Separation-First Multi-Stream Audio Watermarking Feasible via Joint Training

TL;DR

This paper proposes a joint training approach for multi-stream audio watermarking that enables effective watermark recovery after source separation, addressing robustness issues with traditional methods.

Contribution

It introduces an end-to-end training framework that jointly optimizes watermark embedding and source separation, improving post-separation watermark recovery.

Findings

Significant improvement in watermark recovery after separation.

Maintains perceptual audio quality.

Robustness to separation artifacts achieved.

Abstract

Modern audio is created by mixing stems from different sources, raising the question: can we independently watermark each stem and recover all watermarks after separation? We study a separation-first, multi-stream watermarking framework-embedding distinct information into stems using unique keys but a shared structure, mixing, separating, and decoding from each output. A naive pipeline (robust watermarking + off-the-shelf separation) yields poor bit recovery, showing robustness to generic distortions does not ensure robustness to separation artifacts. To enable this, we jointly train the watermark system and the separator in an end-to-end manner, encouraging the separator to preserve watermark cues while adapting embedding to separation-specific distortions. Experiments on speech+music and vocal+accompaniment mixtures show substantial gains in post-separation recovery while maintaining perceptual quality.