De-mark: Watermark Removal in Large Language Models

TL;DR

De-mark is a novel framework that effectively removes n-gram-based watermarks from large language models using a new querying strategy, enhancing the ability to detect machine-generated content.

Contribution

This paper introduces De-mark, a new method for removing watermarks from language models, addressing the robustness gap in watermarking schemes.

Findings

De-mark successfully removes watermarks from Llama3 and ChatGPT.

The random selection probing strategy improves watermark detection and removal.

Experiments show De-mark's efficiency and effectiveness in watermark exploitation.

Abstract

Watermarking techniques offer a promising way to identify machine-generated content via embedding covert information into the contents generated from language models (LMs). However, the robustness of the watermarking schemes has not been well explored. In this paper, we present De-mark, an advanced framework designed to remove n-gram-based watermarks effectively. Our method utilizes a novel querying strategy, termed random selection probing, which aids in assessing the strength of the watermark and identifying the red-green list within the n-gram watermark. Experiments on popular LMs, such as Llama3 and ChatGPT, demonstrate the efficiency and effectiveness of De-mark in watermark removal and exploitation tasks.