ToolGen: Unified Tool Retrieval and Calling via Generation

TL;DR

ToolGen introduces a novel approach where tools are embedded as tokens within large language models, enabling seamless, scalable, and autonomous tool invocation directly through language generation, significantly improving AI task performance.

Contribution

It presents ToolGen, a method that embeds tools into LLMs as tokens, eliminating retrieval steps and enhancing tool utilization and task execution capabilities.

Findings

Achieves superior tool retrieval and task completion results.

Enables access to over 47,000 tools without additional retrieval.

Facilitates end-to-end tool learning and integration with advanced techniques.

Abstract

As large language models (LLMs) advance, their inability to autonomously execute tasks by directly interacting with external tools remains a critical limitation. Traditional methods rely on inputting tool descriptions as context, which is constrained by context length and requires separate, often inefficient, retrieval mechanisms. We introduce ToolGen, a paradigm shift that integrates tool knowledge directly into the LLM's parameters by representing each tool as a unique token. This enables the LLM to generate tool calls and arguments as part of its next token prediction capabilities, seamlessly blending tool invocation with language generation. Our framework allows the LLM to access and utilize a vast amount of tools with no additional retrieval step, significantly enhancing both performance and scalability. Experimental results with over 47,000 tools show that ToolGen not only achieves superior results in both tool retrieval and autonomous task completion but also sets the stage for a new era of AI agents that can adapt to tools across diverse domains. By fundamentally transforming tool retrieval into a generative process, ToolGen paves the way for more versatile, efficient, and autonomous AI systems. ToolGen enables end-to-end tool learning and opens opportunities for integration with other advanced techniques such as chain-of-thought and reinforcement learning, thereby expanding the practical capabilities of LLMs.