SVIP: Towards Verifiable Inference of Open-source Large Language Models

TL;DR

SVIP is a novel protocol enabling users to verify the honesty of decentralized LLM inference providers by using secret-based verification of model representations, ensuring trustworthiness without heavy cryptographic assumptions.

Contribution

We propose SVIP, a computationally efficient, secret-based verification protocol for open-source LLM inference that does not rely on cryptographic assumptions and effectively detects dishonest providers.

Findings

SVIP achieves false negative rates below 5%.

SVIP has false positive rates below 3%.

Verification takes less than 0.01 seconds per prompt.

Abstract

The ever-increasing size of open-source Large Language Models (LLMs) renders local deployment impractical for individual users. Decentralized computing has emerged as a cost-effective solution, allowing individuals and small companies to perform LLM inference for users using surplus computational power. However, a computing provider may stealthily substitute the requested LLM with a smaller, less capable model without consent from users, thereby benefiting from cost savings. We introduce SVIP, a secret-based verifiable LLM inference protocol. Unlike existing solutions based on cryptographic or game-theoretic techniques, our method is computationally effective and does not rest on strong assumptions. Our protocol requires the computing provider to return both the generated text and processed hidden representations from LLMs. We then train a proxy task on these representations, effectively transforming them into a unique model identifier. With our protocol, users can reliably verify whether the computing provider is acting honestly. A carefully integrated secret mechanism further strengthens its security. We thoroughly analyze our protocol under multiple strong and adaptive adversarial scenarios. Our extensive experiments demonstrate that SVIP is accurate, generalizable, computationally efficient, and resistant to various attacks. Notably, SVIP achieves false negative rates below 5% and false positive rates below 3%, while requiring less than 0.01 seconds per prompt query for verification.