Query Routing for Homogeneous Tools: An Instantiation in the RAG Scenario

TL;DR

This paper introduces a method for selecting and routing homogeneous tools based on predicted performance and cost, optimizing task execution in a cost-effective manner.

Contribution

It presents a novel approach to query routing that considers both performance and cost for homogeneous tools, improving efficiency over existing methods.

Findings

Higher performance at lower cost compared to baselines

Effective prediction of tool performance and cost

Cost-effective query assignment

Abstract

Current research on tool learning primarily focuses on selecting the most effective tool from a wide array of options, often overlooking cost-effectiveness, a crucial factor in human problem-solving. In this paper, we address the selection of homogeneous tools by predicting both their performance and the associated cost required to accomplish a given task. We then assign queries to the optimal tools in a cost-effective manner. Our experimental results demonstrate that our method achieves higher performance at a lower cost compared to strong baseline approaches.