XMark: Reliable Multi-Bit Watermarking for LLM-Generated Texts

TL;DR

XMark is a new watermarking method for LLM-generated texts that reliably encodes binary messages while maintaining high text quality, even with limited tokens, outperforming previous approaches.

Contribution

The paper introduces XMark, a novel encoding and decoding scheme that improves decoding accuracy and preserves text quality in multi-bit watermarking for large language models.

Findings

XMark achieves higher decoding accuracy than prior methods.

XMark maintains text quality while embedding watermarks.

XMark is effective with limited tokens in generated texts.

Abstract

Multi-bit watermarking has emerged as a promising solution for embedding imperceptible binary messages into Large Language Model (LLM)-generated text, enabling reliable attribution and tracing of malicious usage of LLMs. Despite recent progress, existing methods still face key limitations: some become computationally infeasible for large messages, while others suffer from a poor trade-off between text quality and decoding accuracy. Moreover, the decoding accuracy of existing methods drops significantly when the number of tokens in the generated text is limited, a condition that frequently arises in practical usage. To address these challenges, we propose \textsc{XMark}, a novel method for encoding and decoding binary messages in LLM-generated texts. The unique design of \textsc{XMark}'s encoder produces a less distorted logit distribution for watermarked token generation, preserving text quality, and also enables its tailored decoder to reliably recover the encoded message with limited tokens. Extensive experiments across diverse downstream tasks show that \textsc{XMark} significantly improves decoding accuracy while preserving the quality of watermarked text, outperforming prior methods. The code is at https://github.com/JiiahaoXU/XMark.