BanglaLorica: Design and Evaluation of a Robust Watermarking Algorithm for Large Language Models in Bangla Text Generation

TL;DR

This paper evaluates and improves watermarking techniques for Bangla LLM-generated text, addressing robustness issues under translation attacks by proposing a layered watermarking strategy that enhances detection accuracy in low-resource language settings.

Contribution

It systematically assesses existing watermarking methods for Bangla, identifies their vulnerabilities under translation attacks, and introduces a layered watermarking approach that significantly improves robustness without retraining.

Findings

KGW and EXP achieve >88% detection accuracy under benign conditions.

RTT attacks reduce detection accuracy to 9-13%.

Layered watermarking improves post-RTT detection accuracy by 25-35%.

Abstract

As large language models (LLMs) are increasingly deployed for text generation, watermarking has become essential for authorship attribution, intellectual property protection, and misuse detection. While existing watermarking methods perform well in high-resource languages, their robustness in low-resource languages remains underexplored. This work presents the first systematic evaluation of state-of-the-art text watermarking methods: KGW, Exponential Sampling (EXP), and Waterfall, for Bangla LLM text generation under cross-lingual round-trip translation (RTT) attacks. Under benign conditions, KGW and EXP achieve high detection accuracy (>88%) with negligible perplexity and ROUGE degradation. However, RTT causes detection accuracy to collapse below RTT causes detection accuracy to collapse to 9-13%, indicating a fundamental failure of token-level watermarking. To address this, we propose a layered watermarking strategy that combines embedding-time and post-generation watermarks. Experimental results show that layered watermarking improves post-RTT detection accuracy by 25-35%, achieving 40-50% accuracy, representing a 3$\times$ to 4$\times$ relative improvement over single-layer methods, at the cost of controlled semantic degradation. Our findings quantify the robustness-quality trade-off in multilingual watermarking and establish layered watermarking as a practical, training-free solution for low-resource languages such as Bangla. Our code and data will be made public.