Ghost in the Transformer: Detecting Model Reuse with Invariant Spectral Signatures

TL;DR

GhostSpec is a novel, data-free method that uses spectral signatures of attention weights to verify the lineage of large language models, ensuring intellectual property protection and model provenance.

Contribution

It introduces GhostSpec, a lightweight, robust, and non-invasive spectral signature technique for verifying LLM lineage without access to training data.

Findings

Robust to fine-tuning, pruning, and adversarial transformations

Efficient and requires minimal computational overhead

Effective in real-world model verification scenarios

Abstract

Large Language Models (LLMs) are widely adopted, but their high training cost leads many developers to fine-tune existing open-source models. While most adhere to open-source licenses, some falsely claim original training despite clear derivation from public models, raising pressing concerns about intellectual property protection and the need to verify model provenance. In this paper, we propose GhostSpec, a lightweight yet effective method for verifying LLM lineage without access to training data or modification of model behavior. Our approach constructs compact and robust fingerprints by applying singular value decomposition (SVD) to invariant products of internal attention weight matrices. Unlike watermarking or output-based methods, GhostSpec is fully data-free, non-invasive, and computationally efficient. Extensive experiments show it is robust to fine-tuning, pruning, expansion, and adversarial transformations, reliably tracing lineage with minimal overhead. By offering a practical solution for model verification, our method contributes to intellectual property protection and fosters a transparent, trustworthy LLM ecosystem. Our code is available at https://github.com/DX0369/GhostSpec.