Truthful Reverse Auctions for Adaptive Selection via Contextual Multi-Armed Bandits

TL;DR

This paper introduces a novel mechanism for adaptive, truthful selection of large language models in reverse auctions, combining mechanism design with contextual multi-armed bandit learning to optimize query-dependent model choice.

Contribution

It develops a new truthful reverse auction mechanism with a contextual MAB algorithm for adaptive LLM selection, addressing gaps in existing auction and learning frameworks.

Findings

Proposes a resampling-based truthful reverse auction mechanism.

Designs a contextual MAB algorithm with sublinear regret.

Unifies mechanism design with adaptive learning for LLM selection.

Abstract

We study the problem of selecting large language models (LLMs) for user queries in settings where multiple LLM providers submit the cost of solving a query. From the users' perspective, choosing an optimal model is a sequential, query-dependent decision problem: high-capacity models offer more reliable outputs but are costlier, while lightweight models are faster and cheaper. We formalize this interaction as a reverse auction design problem with contextual online learning, where the user adaptively discovers which model performs best while eliciting costs from competing LLM providers. Existing multi-armed bandit (MAB) mechanisms focus on forward auctions and social welfare, leaving open the challenges of reverse auctions, provider-optimal outcomes, and contextual adaptation. We address these gaps by designing a resampling-based procedure that generalizes truthful forward MAB mechanisms to reverse auctions and prove that any monotone allocation rule with this procedure is truthful. Using this, we propose a contextual MAB algorithm that learns query-dependent model quality with sublinear regret. Our framework unifies mechanism design and adaptive learning, enabling efficient, truthful, and query-aware LLM selection.