Doc2Agent: Scalable Generation of Tool-Using Agents from API Documentation

TL;DR

Doc2Agent is a scalable pipeline that automatically generates and refines executable tools from unstructured API documentation, enabling web agents to effectively utilize complex real-world APIs with improved performance and lower costs.

Contribution

The paper introduces Doc2Agent, a novel scalable pipeline for converting unstructured API documentation into executable tools for tool-using agents, with iterative refinement and broad applicability.

Findings

Achieved 55% performance improvement on WebArena benchmark.

Reduced API calling costs by 90%.

Demonstrated adaptability to complex, knowledge-rich domains.

Abstract

REST APIs play important roles in enriching the action space of web agents, yet most API-based agents rely on curated and uniform toolsets that do not reflect the complexity of real-world APIs. Building tool-using agents for arbitrary domains remains a major challenge, as it requires reading unstructured API documentation, testing APIs and inferring correct parameters. We propose Doc2Agent, a scalable pipeline to build agents that can call Python-based tools generated from API documentation. Doc2Agent generates executable tools from API documentations and iteratively refines them using a code agent. We evaluate our approach on real-world APIs, WebArena APIs, and research APIs, producing validated tools. We achieved a 55\% relative performance improvement with 90\% lower cost compared to direct API calling on WebArena benchmark. A domain-specific agent built for glycomaterial science further demonstrates the pipeline's adaptability to complex, knowledge-rich tasks. Doc2Agent offers a generalizable solution for building tool agents from unstructured API documentation at scale.