FANTAstic SEquences and Where to Find Them: Faithful and Efficient API Call Generation through State-tracked Constrained Decoding and Reranking

TL;DR

FANTASE introduces a novel approach combining state-tracked constrained decoding and reranking to improve the faithfulness, efficiency, and accuracy of API call generation by large language models.

Contribution

It presents a new output-side optimization method with dynamic API constraint incorporation and a lightweight reranker, enhancing API call generation performance.

Findings

Improves API call accuracy on benchmark datasets.

Enhances inference and context efficiency.

Outperforms existing methods in faithfulness and efficiency.

Abstract

API call generation is the cornerstone of large language models' tool-using ability that provides access to the larger world. However, existing supervised and in-context learning approaches suffer from high training costs, poor data efficiency, and generated API calls that can be unfaithful to the API documentation and the user's request. To address these limitations, we propose an output-side optimization approach called FANTASE. Two of the unique contributions of FANTASE are its State-Tracked Constrained Decoding (SCD) and Reranking components. SCD dynamically incorporates appropriate API constraints in the form of Token Search Trie for efficient and guaranteed generation faithfulness with respect to the API documentation. The Reranking component efficiently brings in the supervised signal by leveraging a lightweight model as the discriminator to rerank the beam-searched candidate generations of the large language model. We demonstrate the superior performance of FANTASE in API call generation accuracy, inference efficiency, and context efficiency with DSTC8 and API Bank datasets.