Waterfall: Framework for Robust and Scalable Text Watermarking and Provenance for LLMs

TL;DR

Waterfall is a novel, training-free framework that enhances the robustness and scalability of text watermarking across various text types and languages, enabling effective IP protection and data provenance for LLMs.

Contribution

It introduces the first training-free, scalable text watermarking framework using LLMs as paraphrasers, improving robustness and applicability for IP protection and LLM data provenance.

Findings

Outperforms state-of-the-art watermarking methods in scalability and robustness.

Effective in watermarking articles and code across multiple languages.

Enables detection of unauthorized LLM training data usage.

Abstract

Protecting intellectual property (IP) of text such as articles and code is increasingly important, especially as sophisticated attacks become possible, such as paraphrasing by large language models (LLMs) or even unauthorized training of LLMs on copyrighted text to infringe such IP. However, existing text watermarking methods are not robust enough against such attacks nor scalable to millions of users for practical implementation. In this paper, we propose Waterfall, the first training-free framework for robust and scalable text watermarking applicable across multiple text types (e.g., articles, code) and languages supportable by LLMs, for general text and LLM data provenance. Waterfall comprises several key innovations, such as being the first to use LLM as paraphrasers for watermarking along with a novel combination of techniques that are surprisingly effective in achieving robust verifiability and scalability. We empirically demonstrate that Waterfall achieves significantly better scalability, robust verifiability, and computational efficiency compared to SOTA article-text watermarking methods, and showed how it could be directly applied to the watermarking of code. We also demonstrated that Waterfall can be used for LLM data provenance, where the watermarks of LLM training data can be detected in LLM output, allowing for detection of unauthorized use of data for LLM training and potentially enabling model-centric watermarking of open-sourced LLMs which has been a limitation of existing LLM watermarking works. Our code is available at https://github.com/aoi3142/Waterfall.