An approach for API synthesis using large language models

TL;DR

This paper introduces a novel API synthesis method leveraging large language models, demonstrating superior performance over existing tools on real-world tasks by effectively capturing developer insights.

Contribution

The paper presents a new approach using large language models for component-based API synthesis, improving efficiency and effectiveness over traditional search-based methods.

Findings

Successfully completed 133 out of 135 real-world tasks

Outperformed the state-of-the-art tool FrAngel in experiments

Demonstrated the potential of LLMs for program synthesis tasks

Abstract

APIs play a pivotal role in modern software development by enabling seamless communication and integration between various systems, applications, and services. Component-based API synthesis is a form of program synthesis that constructs an API by assembling predefined components from a library. Existing API synthesis techniques typically implement dedicated search strategies over bounded spaces of possible implementations, which can be very large and time consuming to explore. In this paper, we present a novel approach of using large language models (LLMs) in API synthesis. LLMs offer a foundational technology to capture developer insights and provide an ideal framework for enabling more effective API synthesis. We perform an experimental evaluation of our approach using 135 real-world programming tasks, and compare it with FrAngel, a state-of-the-art API synthesis tool. The experimental results show that our approach completes 133 of the tasks, and overall outperforms FrAngel. We believe LLMs provide a very useful foundation for tackling the problem of API synthesis, in particular, and program synthesis, in general.