From Summary to Action: Enhancing Large Language Models for Complex Tasks with Open World APIs

TL;DR

This paper introduces Sum2Act, a novel pipeline that enhances large language models' ability to invoke real-world APIs for complex tasks, outperforming existing methods on the ToolBench benchmark.

Contribution

The paper presents a new tool invocation pipeline, Sum2Act, that guides LLMs through summarizing results and deciding actions, improving performance on real-world tasks.

Findings

Sum2Act outperforms ReAct and DFSDT on ToolBench benchmark.

The pipeline effectively controls real-world APIs for complex tasks.

Empirical results show significant performance improvements.

Abstract

The distinction between humans and animals lies in the unique ability of humans to use and create tools. Tools empower humans to overcome physiological limitations, fostering the creation of magnificent civilizations. Similarly, enabling foundational models like Large Language Models (LLMs) with the capacity to learn external tool usage may serve as a pivotal step toward realizing artificial general intelligence. Previous studies in this field have predominantly pursued two distinct approaches to augment the tool invocation capabilities of LLMs. The first approach emphasizes the construction of relevant datasets for model fine-tuning. The second approach, in contrast, aims to fully exploit the inherent reasoning abilities of LLMs through in-context learning strategies. In this work, we introduce a novel tool invocation pipeline designed to control massive real-world APIs. This pipeline mirrors the human task-solving process, addressing complicated real-life user queries. At each step, we guide LLMs to summarize the achieved results and determine the next course of action. We term this pipeline `from Summary to action', Sum2Act for short. Empirical evaluations of our Sum2Act pipeline on the ToolBench benchmark show significant performance improvements, outperforming established methods like ReAct and DFSDT. This highlights Sum2Act's effectiveness in enhancing LLMs for complex real-world tasks.