Pay for The Second-Best Service: A Game-Theoretic Approach Against Dishonest LLM Providers

TL;DR

This paper introduces a game-theoretic model and mechanism design to address dishonest behaviors by LLM API providers, ensuring users can achieve near-optimal utility despite strategic manipulation.

Contribution

It presents the first formal economic model for user-provider interactions with strategic providers and proposes an incentive-compatible mechanism with provable guarantees.

Findings

Existence of an approximate incentive-compatible mechanism with bounded utility loss

Impossibility result showing no mechanism can do asymptotically better

Simulation experiments validate mechanism effectiveness in real API settings

Abstract

The widespread adoption of Large Language Models (LLMs) through Application Programming Interfaces (APIs) induces a critical vulnerability: the potential for dishonest manipulation by service providers. This manipulation can manifest in various forms, such as secretly substituting a proclaimed high-performance model with a low-cost alternative, or inflating responses with meaningless tokens to increase billing. This work tackles the issue through the lens of algorithmic game theory and mechanism design. We are the first to propose a formal economic model for a realistic user-provider ecosystem, where a user can iteratively delegate $T$ queries to multiple model providers, and providers can engage in a range of strategic behaviors. As our central contribution, we prove that for a continuous strategy space and any $\epsilon\in(0,\frac12)$, there exists an approximate incentive-compatible mechanism with an additive approximation ratio of $O(T^{1-\epsilon}\log T)$, and a guaranteed quasi-linear second-best user utility. We also prove an impossibility result, stating that no mechanism can guarantee an expected user utility that is asymptotically better than our mechanism. Furthermore, we demonstrate the effectiveness of our mechanism in simulation experiments with real-world API settings.