Functional Subspace Watermarking for Large Language Models

TL;DR

This paper introduces Functional Subspace Watermarking (FSW), a novel method for embedding ownership signals into large language models that remains robust against common model modifications like fine-tuning and quantization.

Contribution

We propose FSW, a framework that embeds watermarks into a stable functional subspace, balancing robustness and model utility through spectral truncation and a vector consistency constraint.

Findings

FSW achieves higher detection accuracy than existing methods.

The method maintains robustness under various model attacks.

Watermarks do not significantly affect model performance.

Abstract

Model watermarking utilizes internal representations to protect the ownership of large language models (LLMs). However, these features inevitably undergo complex distortions during realistic model modifications such as fine-tuning, quantization, or knowledge distillation, making reliable extraction extremely challenging. Despite extensive research on model-side watermarking, existing methods still lack sufficient robustness against parameter-level perturbations. To address this gap, we propose \texttt{\textbf{Functional Subspace Watermarking (FSW)}}, a framework that anchors ownership signals into a low-dimensional functional backbone. Specifically, we first solve a generalized eigenvalue problem to extract a stable functional subspace for watermark injection, while introducing an adaptive spectral truncation strategy to achieve an optimal balance between robustness and model utility. Furthermore, a vector consistency constraint is incorporated to ensure that watermark injection does not compromise the original semantic performance. Extensive experiments across various LLM architectures and datasets demonstrate that our method achieves superior detection accuracy and statistical verifiability under multiple model attacks, maintaining robustness that outperforms existing state-of-the-art (SOTA) methods.