SELF: A Robust Singular Value and Eigenvalue Approach for LLM Fingerprinting

TL;DR

SELF introduces a novel weight-based fingerprinting method for LLMs using singular value and eigenvalue decomposition, providing robust IP protection resistant to common model modifications and attacks.

Contribution

The paper presents a new intrinsic fingerprinting scheme for LLMs that is transformation-invariant and resistant to false claims, improving upon existing behavior-based and structural methods.

Findings

High detection accuracy for IP infringement

Strong robustness against quantization, pruning, and fine-tuning

Effective few-shot learning-based fingerprint comparison

Abstract

The protection of Intellectual Property (IP) in Large Language Models (LLMs) represents a critical challenge in contemporary AI research. While fingerprinting techniques have emerged as a fundamental mechanism for detecting unauthorized model usage, existing methods -- whether behavior-based or structural -- suffer from vulnerabilities such as false claim attacks or susceptible to weight manipulations. To overcome these limitations, we propose SELF, a novel intrinsic weight-based fingerprinting scheme that eliminates dependency on input and inherently resists false claims. SELF achieves robust IP protection through two key innovations: 1) unique, scalable and transformation-invariant fingerprint extraction via singular value and eigenvalue decomposition of LLM attention weights, and 2) effective neural network-based fingerprint similarity comparison based on few-shot learning and data augmentation. Experimental results demonstrate SELF maintains high IP infringement detection accuracy while showing strong robustness against various downstream modifications, including quantization, pruning, and fine-tuning attacks. Our code is available at https://github.com/HanxiuZhang/SELF_v2.