Attacks and Defenses Against LLM Fingerprinting

TL;DR

This paper explores methods to identify large language models through fingerprinting attacks and proposes defenses that obfuscate model identity while maintaining output quality, enhancing privacy and security.

Contribution

It introduces reinforcement learning-based attack strategies and semantic-preserving defenses for LLM fingerprinting, advancing both offensive and defensive techniques.

Findings

Reinforcement learning improves fingerprinting accuracy with fewer queries.

Semantic filtering reduces fingerprinting success while preserving output quality.

The methods demonstrate practical effectiveness against multiple LLMs.

Abstract

As large language models are increasingly deployed in sensitive environments, fingerprinting attacks pose significant privacy and security risks. We present a study of LLM fingerprinting from both offensive and defensive perspectives. Our attack methodology uses reinforcement learning to automatically optimize query selection, achieving better fingerprinting accuracy with only 3 queries compared to randomly selecting 3 queries from the same pool. Our defensive approach employs semantic-preserving output filtering through a secondary LLM to obfuscate model identity while maintaining semantic integrity. The defensive method reduces fingerprinting accuracy across tested models while preserving output quality. These contributions show the potential to improve fingerprinting tools capabilities while providing practical mitigation strategies against fingerprinting attacks.