Is Your LLM Overcharging You? Tokenization, Transparency, and Incentives
Ander Artola Velasco, Stratis Tsirtsis, Nastaran Okati, Manuel Gomez-Rodriguez

TL;DR
This paper reveals that current token-based pricing for large language models incentivizes providers to overcharge users through strategic misreporting, and proposes a linear character-based pricing mechanism to eliminate this incentive.
Contribution
The study demonstrates the vulnerability of token-based pricing to strategic overcharging and introduces a simple, incentive-compatible pricing scheme based on character count.
Findings
Providers can significantly overcharge without detection using heuristic algorithms.
Linear character-based pricing removes the financial incentive to misreport tokens.
Experimental results with Llama, Gemma, and Ministral models support the theoretical analysis.
Abstract
State-of-the-art large language models require specialized hardware and substantial energy to operate. As a consequence, cloud-based services that provide access to large language models have become very popular. In these services, the price users pay for an output provided by a model depends on the number of tokens the model uses to generate it: they pay a fixed price per token. In this work, we show that this pricing mechanism creates a financial incentive for providers to strategize and misreport the (number of) tokens a model used to generate an output, and users cannot prove, or even know, whether a provider is overcharging them. However, we also show that, if an unfaithful provider is obliged to be transparent about the generative process used by the model, misreporting optimally without raising suspicion is hard. Nevertheless, as a proof-of-concept, we develop an efficient…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsBig Data and Digital Economy · Natural Language Processing Techniques · Mobile Crowdsensing and Crowdsourcing
