Adaptive and Robust Cost-Aware Proof of Quality for Decentralized LLM Inference Networks

TL;DR

This paper enhances decentralized LLM inference networks by developing an adversary-resistant, cost-aware Proof of Quality mechanism that improves consensus robustness against malicious evaluators and strategic attacks.

Contribution

It introduces robust aggregation methods and adaptive trust weighting to improve consensus reliability in decentralized LLM evaluation networks.

Findings

Robust aggregation methods outperform simple averaging in adversarial settings.

Evaluator reliability varies significantly and can invert correlations with true quality.

Larger evaluator sets reduce reward variance but increase operational costs.

Abstract

Decentralized large language model inference networks require lightweight mechanisms to reward high quality outputs under heterogeneous latency and cost. Proof of Quality provides scalable verification by sampling evaluator nodes that score candidate outputs, then aggregating their scores into a consensus signal that determines rewards. However, evaluator heterogeneity and malicious score manipulation can distort consensus and inflate payouts, which weakens incentive alignment in open participation settings. This paper extends a cost-aware Proof of Quality mechanism by adding adversary-resilient consensus formation. We study robust aggregation rules, including median and trimmed mean, and an adaptive trust-weighted consensus that updates evaluator weights from deviation signals. Using question answering and summarization workloads with a ground truth proxy for offline analysis, we quantify evaluator reliability and show strong variance across evaluators, including task-dependent misalignment that can invert correlations. We then evaluate robustness under four adversarial strategies, including noise injection, boosting, sabotage, and intermittent manipulation, across a sweep of malicious ratios and evaluator sample sizes. Our results show that robust aggregation improves consensus alignment with the ground truth proxy and reduces sensitivity to noisy and strategic attacks compared with simple averaging. We further characterize the operational trade-off introduced by evaluator sampling, where larger evaluator sets reduce evaluator rewards and increase payoff variance while inference rewards remain relatively stable in our configuration. These findings motivate robust consensus as a default component for cost-aware Proof of Quality and provide practical guidance for selecting evaluator sampling parameters under adversarial risk and resource constraints.