OASBuilder: Generating OpenAPI Specifications from Online API Documentation with Large Language Models

TL;DR

OASBuilder is a framework that uses large language models and rule-based algorithms to automatically convert diverse online API documentation into standardized OpenAPI specifications, saving manual effort.

Contribution

It introduces a novel pipeline combining LLMs and domain knowledge to generate accurate API specs from unstructured documentation.

Findings

Generalizes well across hundreds of APIs

Produces valid OpenAPI specifications capturing most documentation details

Saves thousands of hours of manual effort in enterprise settings

Abstract

AI agents and business automation tools interacting with external web services require standardized, machine-readable information about their APIs in the form of API specifications. However, the information about APIs available online is often presented as unstructured, free-form HTML documentation, requiring external users to spend significant time manually converting it into a structured format. To address this, we introduce OASBuilder, a novel framework that transforms long and diverse API documentation pages into consistent, machine-readable API specifications. This is achieved through a carefully crafted pipeline that integrates large language models and rule-based algorithms which are guided by domain knowledge of the structure of documentation webpages. Our experiments demonstrate that OASBuilder generalizes well across hundreds of APIs, and produces valid OpenAPI specifications that encapsulate most of the information from the original documentation. OASBuilder has been successfully implemented in an enterprise environment, saving thousands of hours of manual effort and making hundreds of complex enterprise APIs accessible as tools for LLMs.