Adaptive Text Watermark for Large Language Models

TL;DR

This paper introduces an adaptive watermarking method for large language models that enhances text quality, security, and robustness without prior knowledge of prompts, using entropy-based token selection and semantic scaling.

Contribution

It proposes a novel adaptive watermarking strategy that dynamically adjusts watermark embedding based on token entropy and semantic context, improving over fixed methods.

Findings

Achieves comparable robustness to existing watermark techniques.

Maintains low perplexity similar to unwatermarked models.

Remains secure under various attack scenarios.

Abstract

The advancement of Large Language Models (LLMs) has led to increasing concerns about the misuse of AI-generated text, and watermarking for LLM-generated text has emerged as a potential solution. However, it is challenging to generate high-quality watermarked text while maintaining strong security, robustness, and the ability to detect watermarks without prior knowledge of the prompt or model. This paper proposes an adaptive watermarking strategy to address this problem. To improve the text quality and maintain robustness, we adaptively add watermarking to token distributions with high entropy measured using an auxiliary model and keep the low entropy token distributions untouched. For the sake of security and to further minimize the watermark's impact on text quality, instead of using a fixed green/red list generated from a random secret key, which can be vulnerable to decryption and forgery, we adaptively scale up the output logits in proportion based on the semantic embedding of previously generated text using a well designed semantic mapping model. Our experiments involving various LLMs demonstrate that our approach achieves comparable robustness performance to existing watermark methods. Additionally, the text generated by our method has perplexity comparable to that of \emph{un-watermarked} LLMs while maintaining security even under various attacks.