TensorCommitments: A Lightweight Verifiable Inference for Language Models

TL;DR

This paper introduces TensorCommitments, a cryptographic scheme enabling lightweight, verifiable inference for large language models, ensuring correctness without significant computational overhead or reliance on powerful verifier hardware.

Contribution

The paper presents TensorCommitments, a novel tensor-native proof-of-inference scheme that efficiently verifies LLM inferences with minimal overhead and enhanced robustness against tampering.

Findings

Adds less than 1% prover time and 0.12% verifier time for LLaMA2 inference.

Improves robustness to tailored LLM attacks by up to 48%.

Achieves verifiable inference without requiring a strong verifier GPU.

Abstract

Most large language models (LLMs) run on external clouds: users send a prompt, pay for inference, and must trust that the remote GPU executes the LLM without any adversarial tampering. We critically ask how to achieve verifiable LLM inference, where a prover (the service) must convince a verifier (the client) that an inference was run correctly without rerunning the LLM. Existing cryptographic works are too slow at the LLM scale, while non-cryptographic ones require a strong verifier GPU. We propose TensorCommitments (TCs), a tensor-native proof-of-inference scheme. TC binds the LLM inference to a commitment, an irreversible tag that breaks under tampering, organized in our multivariate Terkle Trees. For LLaMA2, TC adds only 0.97% prover and 0.12% verifier time over inference while improving robustness to tailored LLM attacks by up to 48% over the best prior work requiring a verifier GPU.