Evaluating the Robustness and Accuracy of Text Watermarking Under Real-World Cross-Lingual Manipulations

TL;DR

This paper benchmarks the robustness and accuracy of text watermarking methods across multiple languages, focusing on real-world cross-lingual attacks and practical scenarios often overlooked in prior research.

Contribution

It introduces a comprehensive evaluation of watermarking techniques in cross-lingual contexts, highlighting their effectiveness and vulnerabilities in multilingual and adversarial scenarios.

Findings

Watermarking methods vary in robustness across languages.

Cross-lingual translation attacks can significantly reduce watermark detectability.

Current watermarking techniques may not be reliable in multilingual adversarial settings.

Abstract

We present a study to benchmark representative watermarking methods in cross-lingual settings. The current literature mainly focuses on the evaluation of watermarking methods for the English language. However, the literature for evaluating watermarking in cross-lingual settings is scarce. This results in overlooking important adversary scenarios in which a cross-lingual adversary could be in, leading to a gray area of practicality over cross-lingual watermarking. In this paper, we evaluate four watermarking methods in four different and vocabulary rich languages. Our experiments investigate the quality of text under different watermarking procedure and the detectability of watermarks with practical translation attack scenarios. Specifically, we investigate practical scenarios that an adversary with cross-lingual knowledge could take, and evaluate whether current watermarking methods are suitable for such scenarios. Finally, from our findings, we draw key insights about watermarking in cross-lingual settings.