Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Models

TL;DR

This paper presents a novel multi-objective optimization watermarking method for large language models that improves detectability while preserving the semantic quality of generated texts.

Contribution

It introduces a lightweight, token-specific watermarking approach using multi-objective optimization to enhance both detectability and semantic coherence.

Findings

Outperforms existing watermarking methods in detectability.

Maintains high semantic coherence in watermarked texts.

Demonstrates effectiveness on large language models.

Abstract

Large language models generate high-quality responses with potential misinformation, underscoring the need for regulation by distinguishing AI-generated and human-written texts. Watermarking is pivotal in this context, which involves embedding hidden markers in texts during the LLM inference phase, which is imperceptible to humans. Achieving both the detectability of inserted watermarks and the semantic quality of generated texts is challenging. While current watermarking algorithms have made promising progress in this direction, there remains significant scope for improvement. To address these challenges, we introduce a novel multi-objective optimization (MOO) approach for watermarking that utilizes lightweight networks to generate token-specific watermarking logits and splitting ratios. By leveraging MOO to optimize for both detection and semantic objective functions, our method simultaneously achieves detectability and semantic integrity. Experimental results show that our method outperforms current watermarking techniques in enhancing the detectability of texts generated by LLMs while maintaining their semantic coherence. Our code is available at https://github.com/mignonjia/TS_watermark.