Re-Invoke: Tool Invocation Rewriting for Zero-Shot Tool Retrieval

TL;DR

Re-Invoke is an unsupervised method that improves large-scale tool retrieval for LLM-powered agents by generating synthetic queries, extracting query context, and ranking tools based on intent similarity, outperforming existing methods.

Contribution

It introduces a novel unsupervised approach combining synthetic query generation and multi-view intent-based ranking for scalable tool retrieval without training.

Findings

20% improvement in nDCG@5 for single-tool retrieval

39% improvement in nDCG@5 for multi-tool retrieval

Outperforms state-of-the-art methods in large toolset scenarios

Abstract

Recent advances in large language models (LLMs) have enabled autonomous agents with complex reasoning and task-fulfillment capabilities using a wide range of tools. However, effectively identifying the most relevant tools for a given task becomes a key bottleneck as the toolset size grows, hindering reliable tool utilization. To address this, we introduce Re-Invoke, an unsupervised tool retrieval method designed to scale effectively to large toolsets without training. Specifically, we first generate a diverse set of synthetic queries that comprehensively cover different aspects of the query space associated with each tool document during the tool indexing phase. Second, we leverage LLM's query understanding capabilities to extract key tool-related context and underlying intents from user queries during the inference phase. Finally, we employ a novel multi-view similarity ranking strategy based on intents to pinpoint the most relevant tools for each query. Our evaluation demonstrates that Re-Invoke significantly outperforms state-of-the-art alternatives in both single-tool and multi-tool scenarios, all within a fully unsupervised setting. Notably, on the ToolE datasets, we achieve a 20% relative improvement in nDCG@5 for single-tool retrieval and a 39% improvement for multi-tool retrieval.