GenTool: Enhancing Tool Generalization in Language Models through Zero-to-One and Weak-to-Strong Simulation

TL;DR

GenTool is a training framework that significantly improves large language models' ability to generalize in tool usage, enabling them to handle unseen queries and leverage better tools, thus enhancing their AI assistant capabilities.

Contribution

The paper introduces GenTool, a novel two-stage fine-tuning approach with synthetic data to enhance LLMs' zero-to-one and weak-to-strong tool generalization capabilities.

Findings

Substantially improves tool-usage performance across models from 1B to 8B parameters.

Outperforms GPT-4o in generalization scenarios.

Provides insights into challenges faced by LLMs in tool generalization.

Abstract

Large Language Models (LLMs) can enhance their capabilities as AI assistants by integrating external tools, allowing them to access a wider range of information. While recent LLMs are typically fine-tuned with tool usage examples during supervised fine-tuning (SFT), questions remain about their ability to develop robust tool-usage skills and can effectively generalize to unseen queries and tools. In this work, we present GenTool, a novel training framework that prepares LLMs for diverse generalization challenges in tool utilization. Our approach addresses two fundamental dimensions critical for real-world applications: Zero-to-One Generalization, enabling the model to address queries initially lacking a suitable tool by adopting and utilizing one when it becomes available, and Weak-to-Strong Generalization, allowing models to leverage enhanced versions of existing tools to solve queries. To achieve this, we develop synthetic training data simulating these two dimensions of tool usage and introduce a two-stage fine-tuning approach: optimizing tool ranking, then refining tool selection. Through extensive experiments across four generalization scenarios, we demonstrate that our method significantly enhances the tool-usage capabilities of LLMs ranging from 1B to 8B parameters, achieving performance that surpasses GPT-4o. Furthermore, our analysis also provides valuable insights into the challenges LLMs encounter in tool generalization.