Harnessing the Power of Multiple Minds: Lessons Learned from LLM Routing

TL;DR

This paper investigates the feasibility of directing each query to the most suitable large language model (LLM) for complex reasoning tasks, highlighting potential and limitations of LLM routing.

Contribution

It introduces the concept of LLM routing for challenging reasoning tasks and provides experimental insights into its effectiveness and challenges.

Findings

LLM routing shows promise for complex reasoning tasks.

Routing is not always feasible across all scenarios.

Further research needed for more robust approaches.

Abstract

With the rapid development of LLMs, it is natural to ask how to harness their capabilities efficiently. In this paper, we explore whether it is feasible to direct each input query to a single most suitable LLM. To this end, we propose LLM routing for challenging reasoning tasks. Our extensive experiments suggest that such routing shows promise but is not feasible in all scenarios, so more robust approaches should be investigated to fill this gap.