Baselines for Identifying Watermarked Large Language Models

TL;DR

This paper presents baseline algorithms for detecting watermarks in large language models by analyzing output distributions, highlighting differences between watermarked and unmarked models, and exploring detection tradeoffs.

Contribution

It introduces a formal framework and baseline methods for watermark detection in LLMs, addressing the problem of identifying watermarked models in the wild.

Findings

Watermarked LLMs produce distinguishable output distributions.

Detection effectiveness varies with watermark strength.

Tradeoffs exist between detection accuracy and watermark robustness.

Abstract

We consider the emerging problem of identifying the presence and use of watermarking schemes in widely used, publicly hosted, closed source large language models (LLMs). We introduce a suite of baseline algorithms for identifying watermarks in LLMs that rely on analyzing distributions of output tokens and logits generated by watermarked and unmarked LLMs. Notably, watermarked LLMs tend to produce distributions that diverge qualitatively and identifiably from standard models. Furthermore, we investigate the identifiability of watermarks at varying strengths and consider the tradeoffs of each of our identification mechanisms with respect to watermarking scenario. Along the way, we formalize the specific problem of identifying watermarks in LLMs, as well as LLM watermarks and watermark detection in general, providing a framework and foundations for studying them.