Close the Loop: Synthesizing Infinite Tool-Use Data via Multi-Agent Role-Playing

TL;DR

InfTool is an autonomous multi-agent framework that synthesizes diverse tool-use data for large language models, significantly improving their ability to invoke external tools without human annotation.

Contribution

The paper introduces InfTool, a self-evolving multi-agent system that generates high-quality synthetic data for training LLMs to use external tools, overcoming annotation and generalization challenges.

Findings

Achieved 70.9% accuracy on BFCL, a 258% improvement over the base model.

Surpassed larger models and rivaled Claude-Opus using only synthetic data.

Demonstrated effective closed-loop training without human intervention.

Abstract

Enabling Large Language Models (LLMs) to reliably invoke external tools remains a critical bottleneck for autonomous agents. Existing approaches suffer from three fundamental challenges: expensive human annotation for high-quality trajectories, poor generalization to unseen tools, and quality ceilings inherent in single-model synthesis that perpetuate biases and coverage gaps. We introduce InfTool, a fully autonomous framework that breaks these barriers through self-evolving multi-agent synthesis. Given only raw API specifications, InfTool orchestrates three collaborative agents (User Simulator, Tool-Calling Assistant, and MCP Server) to generate diverse, verified trajectories spanning single-turn calls to complex multi-step workflows. The framework establishes a closed loop: synthesized data trains the model via Group Relative Policy Optimization (GRPO) with gated rewards, the improved model generates higher-quality data targeting capability gaps, and this cycle iterates without human intervention. Experiments on the Berkeley Function-Calling Leaderboard (BFCL) demonstrate that InfTool transforms a base 32B model from 19.8% to 70.9% accuracy (+258%), surpassing models 10x larger and rivaling Claude-Opus, and entirely from synthetic data without human annotation.