Provably Robust Multi-bit Watermarking for AI-generated Text

TL;DR

This paper presents a new provably robust watermarking method for AI-generated text that significantly improves accuracy and robustness in content source tracing, enabling reliable identification of the source even after text edits.

Contribution

The paper introduces a novel watermarking technique based on pseudo-random segment assignment, enhancing robustness and accuracy over existing methods for LLM-generated text.

Findings

Achieves 97.6% match rate for 20-bit messages in 200-token texts

Outperforms existing baselines in accuracy and robustness

Tolerates an average edit distance of 17 per paragraph

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities of generating texts resembling human language. However, they can be misused by criminals to create deceptive content, such as fake news and phishing emails, which raises ethical concerns. Watermarking is a key technique to address these concerns, which embeds a message (e.g., a bit string) into a text generated by an LLM. By embedding the user ID (represented as a bit string) into generated texts, we can trace generated texts to the user, known as content source tracing. The major limitation of existing watermarking techniques is that they achieve sub-optimal performance for content source tracing in real-world scenarios. The reason is that they cannot accurately or efficiently extract a long message from a generated text. We aim to address the limitations. In this work, we introduce a new watermarking method for LLM-generated text grounded in pseudo-random segment assignment. We also propose multiple techniques to further enhance the robustness of our watermarking algorithm. We conduct extensive experiments to evaluate our method. Our experimental results show that our method substantially outperforms existing baselines in both accuracy and robustness on benchmark datasets. For instance, when embedding a message of length 20 into a 200-token generated text, our method achieves a match rate of $97.6\%$, while the state-of-the-art work Yoo et al. only achieves $49.2\%$. Additionally, we prove that our watermark can tolerate edits within an edit distance of 17 on average for each paragraph under the same setting.