EmbTracker: Traceable Black-box Watermarking for Federated Language Models

TL;DR

EmbTracker is a novel server-side black-box watermarking framework for federated language models that enables individual traceability and robust ownership verification without requiring client cooperation.

Contribution

It introduces EmbTracker, the first black-box watermarking scheme for FedLMs that achieves client-level traceability and high robustness against removal attacks.

Findings

Near 100% verification accuracy in experiments

High resilience against fine-tuning, pruning, and quantization

Minimal impact on primary task performance (within 1-2%)

Abstract

Federated Language Model (FedLM) allows a collaborative learning without sharing raw data, yet it introduces a critical vulnerability, as every untrustworthy client may leak the received functional model instance. Current watermarking schemes for FedLM often require white-box access and client-side cooperation, providing only group-level proof of ownership rather than individual traceability. We propose EmbTracker, a server-side, traceable black-box watermarking framework specifically designed for FedLMs. EmbTracker achieves black-box verifiability by embedding a backdoor-based watermark detectable through simple API queries. Client-level traceability is realized by injecting unique identity-specific watermarks into the model distributed to each client. In this way, a leaked model can be attributed to a specific culprit, ensuring robustness even against non-cooperative participants. Extensive experiments on various language and vision-language models demonstrate that EmbTracker achieves robust traceability with verification rates near 100\%, high resilience against removal attacks (fine-tuning, pruning, quantization), and negligible impact on primary task performance (typically within 1-2\%).