Proof of Quality: A Costless Paradigm for Trustless Generative AI Model Inference on Blockchains

TL;DR

This paper introduces 'proof of quality', a blockchain-compatible method for verifying the output quality of large generative AI models, ensuring trustless deployment with minimal overhead and high efficiency.

Contribution

It proposes a novel quality-based inference paradigm for blockchain deployment, using lightweight models for rapid and robust validation of generative AI outputs.

Findings

Validation overhead is minimal, completing within a second on CPU.

PoQ consensus is generated in milliseconds, 1,000 times faster than existing schemes.

The protocol is robust against rational adversarial participants.

Abstract

Generative AI models, such as GPT-4 and Stable Diffusion, have demonstrated powerful and disruptive capabilities in natural language and image tasks. However, deploying these models in decentralized environments remains challenging. Unlike traditional centralized deployment, systematically guaranteeing the integrity of AI model services in fully decentralized environments, particularly on trustless blockchains, is both crucial and difficult. In this paper, we present a new inference paradigm called \emph{proof of quality} (PoQ) to enable the deployment of arbitrarily large generative models on blockchain architecture. Unlike traditional approaches based on validating inference procedures, such as ZKML or OPML, our PoQ paradigm focuses on the outcome quality of model inference. Using lightweight BERT-based cross-encoders as our underlying quality evaluation model, we design and implement PQML, the first practical protocol for real-world NLP generative model inference on blockchains, tailored for popular open-source models such as Llama 3 and Mixtral. Our analysis demonstrates that our protocol is robust against adversarial but rational participants in ecosystems, where lazy or dishonest behavior results in fewer benefits compared to well-behaving participants. The computational overhead of validating the quality evaluation is minimal, allowing quality validators to complete the quality check within a second, even using only a CPU. Preliminary simulation results show that PoQ consensus is generated in milliseconds, 1,000 times faster than any existing scheme.