SoK: Large Language Model Copyright Auditing via Fingerprinting

TL;DR

This paper provides a comprehensive survey and evaluation framework for LLM fingerprinting techniques, addressing their reliability and robustness in copyright auditing through a new benchmark and taxonomy.

Contribution

It introduces the first unified framework and taxonomy for LLM fingerprinting methods and proposes LeaFBench, a systematic benchmark for evaluating these methods under realistic scenarios.

Findings

Existing fingerprinting methods have varying strengths and weaknesses.

LeaFBench enables systematic evaluation of fingerprinting techniques.

The study highlights open challenges and future directions in LLM copyright auditing.

Abstract

The broad capabilities and substantial resources required to train Large Language Models (LLMs) make them valuable intellectual property, yet they remain vulnerable to copyright infringement, such as unauthorized use and model theft. LLM fingerprinting, a non-intrusive technique that compares the distinctive features (i.e., fingerprint) of LLMs to identify whether an LLM is derived from another, offers a promising solution to copyright auditing. However, its reliability remains uncertain due to the prevalence of diverse model modifications and the lack of standardized evaluation. In this SoK, we present the first comprehensive study of the emerging LLM fingerprinting. We introduce a unified framework and taxonomy that structures the field: white-box methods are classified based on their feature source as static, forward-pass, or backward-pass fingerprinting, while black-box methods are distinguished by their query strategy as either untargeted or targeted. Furthermore, we propose LeaFBench, the first systematic benchmark for evaluating LLM fingerprinting under realistic deployment scenarios. Built upon 7 mainstream foundation models and comprising 149 distinct model instances, LeaFBench integrates 13 representative post-development techniques, spanning both parameter-altering methods (e.g., fine-tuning, quantization) and parameter-independent techniques (e.g., system prompts, RAG). Extensive experiments on LeaFBench reveal the strengths and weaknesses of existing methods, thereby outlining future research directions and critical open problems in this emerging field. The code is available at https://github.com/shaoshuo-ss/LeaFBench.