Security and Detectability Analysis of Unicode Text Watermarking Methods Against Large Language Models

TL;DR

This study evaluates the security and detectability of ten Unicode text watermarking methods against six large language models, revealing that advanced models can detect watermarks but cannot extract them without source code access.

Contribution

It provides a comprehensive analysis of existing Unicode text watermarking methods' robustness against modern large language models, highlighting vulnerabilities and security implications.

Findings

Large language models can detect watermarked text.

Watermarks cannot be extracted without source code.

Latest models show increased detection capabilities.

Abstract

Securing digital text is becoming increasingly relevant due to the widespread use of large language models. Individuals' fear of losing control over data when it is being used to train such machine learning models or when distinguishing model-generated output from text written by humans. Digital watermarking provides additional protection by embedding an invisible watermark within the data that requires protection. However, little work has been taken to analyze and verify if existing digital text watermarking methods are secure and undetectable by large language models. In this paper, we investigate the security-related area of watermarking and machine learning models for text data. In a controlled testbed of three experiments, ten existing Unicode text watermarking methods were implemented and analyzed across six large language models: GPT-5, GPT-4o, Teuken 7B, Llama 3.3, Claude Sonnet 4, and Gemini 2.5 Pro. The findings of our experiments indicate that, especially the latest reasoning models, can detect a watermarked text. Nevertheless, all models fail to extract the watermark unless implementation details in the form of source code are provided. We discuss the implications for security researchers and practitioners and outline future research opportunities to address security concerns.