Origin Tracer: A Method for Detecting LoRA Fine-Tuning Origins in LLMs

TL;DR

The paper introduces Origin-Tracer, a novel method for verifying if an LLM was fine-tuned from a specific base model, even under obfuscation, enhancing transparency and trust in model origins.

Contribution

It presents the first formalized approach to detect fine-tuning origins and extract LoRA rank, improving robustness against obfuscation techniques.

Findings

Effectively detects fine-tuning origins under obfuscation scenarios

Successfully extracts LoRA rank during fine-tuning

Establishes new benchmarks for model verification

Abstract

As large language models (LLMs) continue to advance, their deployment often involves fine-tuning to enhance performance on specific downstream tasks. However, this customization is sometimes accompanied by misleading claims about the origins, raising significant concerns about transparency and trust within the open-source community. Existing model verification techniques typically assess functional, representational, and weight similarities. However, these approaches often struggle against obfuscation techniques, such as permutations and scaling transformations. To address this limitation, we propose a novel detection method Origin-Tracer that rigorously determines whether a model has been fine-tuned from a specified base model. This method includes the ability to extract the LoRA rank utilized during the fine-tuning process, providing a more robust verification framework. This framework is the first to provide a formalized approach specifically aimed at pinpointing the sources of model fine-tuning. We empirically validated our method on thirty-one diverse open-source models under conditions that simulate real-world obfuscation scenarios. We empirically analyze the effectiveness of our framework and finally, discuss its limitations. The results demonstrate the effectiveness of our approach and indicate its potential to establish new benchmarks for model verification.