EchoMark: Perceptual Acoustic Environment Transfer with Watermark-Embedded Room Impulse Response

TL;DR

EchoMark is a deep learning framework that transfers acoustic environments with embedded watermarks, enabling high-quality room impulse response generation while ensuring watermark robustness for security.

Contribution

It introduces the first perceptual acoustic environment transfer method with watermark embedding, operating in the latent domain to handle variable RIR characteristics.

Findings

Achieves room acoustic parameter matching comparable to state-of-the-art methods.

Attains a high MOS score of 4.22 out of 5 for perceptual quality.

Watermark detection accuracy exceeds 99%, with BER below 0.3%.

Abstract

Acoustic Environment Matching (AEM) is the task of transferring clean audio into a target acoustic environment, enabling engaging applications such as audio dubbing and auditory immersive virtual reality (VR). Recovering similar room impulse response (RIR) directly from reverberant speech offers more accessible and flexible AEM solution. However, this capability also introduces vulnerabilities of arbitrary ``relocation" if misused by malicious user, such as facilitating advanced voice spoofing attacks or undermining the authenticity of recorded evidence. To address this issue, we propose EchoMark, the first deep learning-based AEM framework that generates perceptually similar RIRs with embedded watermark. Our design tackle the challenges posed by variable RIR characteristics, such as different durations and energy decays, by operating in the latent domain. By jointly optimizing the model with a perceptual loss for RIR reconstruction and a loss for watermark detection, EchoMark achieves both high-quality environment transfer and reliable watermark recovery. Experiments on diverse datasets validate that EchoMark achieves room acoustic parameter matching performance comparable to FiNS, the state-of-the-art RIR estimator. Furthermore, a high Mean Opinion Score (MOS) of 4.22 out of 5, watermark detection accuracy exceeding 99\%, and bit error rates (BER) below 0.3\% collectively demonstrate the effectiveness of EchoMark in preserving perceptual quality while ensuring reliable watermark embedding.