LLM Bandit: Cost-Efficient LLM Generation via Preference-Conditioned Dynamic Routing

TL;DR

This paper introduces a dynamic routing framework for selecting the most suitable large language model for user queries, balancing accuracy and cost through a preference-conditioned bandit approach.

Contribution

It formulates LLM selection as a multi-armed bandit problem with a preference-conditioned routing mechanism, enabling adaptive, cost-effective model selection that generalizes to unseen models.

Findings

Significant improvements in accuracy and cost savings across LLM platforms.

Effective adaptation to user preferences at inference time.

Generalizes well to new, unseen LLMs.

Abstract

The rapid advancement in large language models (LLMs) has brought forth a diverse range of models with varying capabilities that excel in different tasks and domains. However, selecting the optimal LLM for user queries often involves a challenging trade-off between accuracy and cost, a problem exacerbated by the diverse demands of individual queries. In this work, we present a novel framework that formulates the LLM selection process as a multi-armed bandit problem, enabling dynamic and intelligent routing of queries to the most appropriate model. Our approach incorporates a preference-conditioned dynamic routing mechanism, allowing users to specify their preferences at inference time, thereby offering a customizable balance between performance and cost. Additionally, our selection policy is designed to generalize to unseen LLMs, ensuring adaptability to new models as they emerge. Experimental results demonstrate that our method achieves significant improvements in both accuracy and cost-effectiveness across various LLM platforms, showcasing the potential of our framework to adaptively optimize LLM selection in real-world scenarios.