Robust and Minimally Invasive Watermarking for EaaS

TL;DR

This paper introduces ESpeW, a novel embedding-specific watermarking method that provides robust, minimally invasive copyright protection for Embeddings as a Service, resistant to removal attacks while maintaining embedding quality.

Contribution

The paper presents a new watermarking mechanism that injects unique, identifiable watermarks into each embedding, significantly improving robustness against removal and reducing impact on embedding quality.

Findings

ESpeW effectively resists aggressive removal strategies.

Watermarks are injected with less than 1% impact on embeddings.

The method maintains high embedding quality across datasets.

Abstract

Embeddings as a Service (EaaS) is emerging as a crucial role in AI applications. Unfortunately, EaaS is vulnerable to model extraction attacks, highlighting the urgent need for copyright protection. Although some preliminary works propose applying embedding watermarks to protect EaaS, recent research reveals that these watermarks can be easily removed. Hence, it is crucial to inject robust watermarks resistant to watermark removal attacks. Existing watermarking methods typically inject a target embedding into embeddings through linear interpolation when the text contains triggers. However, this mechanism results in each watermarked embedding having the same component, which makes the watermark easy to identify and eliminate. Motivated by this, in this paper, we propose a novel embedding-specific watermarking (ESpeW) mechanism to offer robust copyright protection for EaaS. Our approach involves injecting unique, yet readily identifiable watermarks into each embedding. Watermarks inserted by ESpeW are designed to maintain a significant distance from one another and to avoid sharing common components, thus making it significantly more challenging to remove the watermarks. Moreover, ESpeW is minimally invasive, as it reduces the impact on embeddings to less than 1\%, setting a new milestone in watermarking for EaaS. Extensive experiments on four popular datasets demonstrate that ESpeW can even watermark successfully against a highly aggressive removal strategy without sacrificing the quality of embeddings.