Explore-Construct-Filter: An Automated Framework for Rich and Reliable API Knowledge Graph Construction

TL;DR

This paper introduces an automated framework using large language models to construct a rich and reliable API knowledge graph with minimal manual effort, improving both coverage and trustworthiness.

Contribution

The Explore-Construct-Filter framework automates API knowledge graph construction, reducing manual schema design and enhancing reliability through filtering, outperforming existing methods.

Findings

Achieved 25.2% improvement in F1 score over state-of-the-art.

Increased KG richness by 133.6% through exploration.

Improved reliability by 26.6% via filtering.

Abstract

The API Knowledge Graph (API KG) is a structured network that models API entities and their relations, providing essential semantic insights for tasks such as API recommendation, code generation, and API misuse detection. However, constructing a knowledge-rich and reliable API KG presents several challenges. Existing schema-based methods rely heavily on manual annotations to design KG schemas, leading to excessive manual overhead. On the other hand, schema-free methods, due to the lack of schema guidance, are prone to introducing noise, reducing the KG's reliability. To address these issues, we propose the Explore-Construct-Filter framework, an automated approach for API KG construction based on large language models (LLMs). This framework consists of three key modules: 1) KG exploration: LLMs simulate the workflow of annotators to automatically design a schema with comprehensive type triples, minimizing human intervention; 2) KG construction: Guided by the schema, LLMs extract instance triples to construct a rich yet unreliable API KG; 3) KG filtering: Removing invalid type triples and suspicious instance triples to construct a rich and reliable API KG. Experimental results demonstrate that our method surpasses the state-of-the-art method, achieving a 25.2% improvement in F1 score. Moreover, the Explore-Construct-Filter framework proves effective, with the KG exploration module increasing KG richness by 133.6% and the KG filtering module improving reliability by 26.6%. Finally, cross-model experiments confirm the generalizability of our framework.