IMMACULATE: A Practical LLM Auditing Framework via Verifiable Computation

TL;DR

IMMACULATE is a practical framework for auditing commercial large language models, detecting deviations like model substitution and overbilling without needing internal access or trusted hardware, with minimal performance impact.

Contribution

It introduces a verifiable computation-based auditing method that efficiently detects malicious behaviors in black-box LLM APIs with strong guarantees.

Findings

Reliable detection of deviations with under 1% throughput overhead

Effective distinction between benign and malicious executions

Applicable to dense and MoE models

Abstract

Commercial large language models are typically deployed as black-box API services, requiring users to trust providers to execute inference correctly and report token usage honestly. We present IMMACULATE, a practical auditing framework that detects economically motivated deviations-such as model substitution, quantization abuse, and token overbilling-without trusted hardware or access to model internals. IMMACULATE selectively audits a small fraction of requests using verifiable computation, achieving strong detection guarantees while amortizing cryptographic overhead. Experiments on dense and MoE models show that IMMACULATE reliably distinguishes benign and malicious executions with under 1% throughput overhead. Our code is published at https://github.com/guo-yanpei/Immaculate.