Beyond A Fixed Seal: Adaptive Stealing Watermark in Large Language Models

TL;DR

This paper introduces Adaptive Stealing, a flexible watermark attack method for large language models that dynamically selects attack strategies to improve efficiency against watermarked texts.

Contribution

The paper proposes a novel adaptive stealing algorithm with position-based and dynamic selection modules, enhancing attack effectiveness over fixed strategies.

Findings

AS significantly improves steal efficiency against watermarks.

Dynamic perspective selection outperforms fixed strategies.

Code is publicly available for future research.

Abstract

Watermarking provides a critical safeguard for large language model (LLM) services by facilitating the detection of LLM-generated text. Correspondingly, stealing watermark algorithms (SWAs) derive watermark information from watermarked texts generated by victim LLMs to craft highly targeted adversarial attacks, which compromise the reliability of watermarks. Existing SWAs rely on fixed strategies, overlooking the non-uniform distribution of stolen watermark information and the dynamic nature of real-world LLM generation processes. To address these limitations, we propose Adaptive Stealing (AS), a novel SWA featuring enhanced design flexibility through Position-Based Seal Construction and Adaptive Selection modules. AS operates by defining multiple attack perspectives derived from distinct activation states of contextually ordered tokens. During attack execution, AS dynamically selects the optimal perspective based on watermark compatibility, generation priority, and dynamic generation relevance. Our experiments demonstrate that AS significantly increases steal efficiency against target watermarks under identical experimental conditions. These findings highlight the need for more robust LLM watermarks to withstand potential attacks. We release our code to the community for future research\footnote{https://github.com/DrankXs/AdaptiveStealingWatermark}.