Distortion-free Watermarks are not Truly Distortion-free under Watermark Key Collisions
Yihan Wu, Ruibo Chen, Zhengmian Hu, Yanshuo Chen, Junfeng Guo,, Hongyang Zhang, Heng Huang

TL;DR
This paper investigates the limitations of distortion-free watermarks in language models, revealing that key collisions cause distribution bias and that perfect distortion-free watermarking is impossible, proposing beta-watermark as a mitigation.
Contribution
The paper identifies the impact of key collisions on distortion-free watermarks and introduces beta-watermark to reduce distribution bias under such collisions.
Findings
Key collisions are inevitable due to limited watermark keys.
Existing distortion-free watermarks exhibit distribution bias under key collisions.
Beta-watermark reduces distribution bias effectively.
Abstract
Language model (LM) watermarking techniques inject a statistical signal into LM-generated content by substituting the random sampling process with pseudo-random sampling, using watermark keys as the random seed. Among these statistical watermarking approaches, distortion-free watermarks are particularly crucial because they embed watermarks into LM-generated content without compromising generation quality. However, one notable limitation of pseudo-random sampling compared to true-random sampling is that, under the same watermark keys (i.e., key collision), the results of pseudo-random sampling exhibit correlations. This limitation could potentially undermine the distortion-free property. Our studies reveal that key collisions are inevitable due to the limited availability of watermark keys, and existing distortion-free watermarks exhibit a significant distribution bias toward the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Steganography and Watermarking Techniques · Digital Rights Management and Security · Physical Unclonable Functions (PUFs) and Hardware Security
