Tool Learning with Large Language Models: A Survey

TL;DR

This survey comprehensively reviews the recent advancements in tool learning with large language models, covering motivations, implementation stages, benchmarks, challenges, and future directions to facilitate further research and development.

Contribution

It provides a systematic organization of existing literature on tool learning with LLMs, including a taxonomy of workflow stages and evaluation methods, which was previously fragmented.

Findings

Identified key benefits of tool learning with LLMs.

Categorized the workflow into four stages: task planning, tool selection, tool calling, response generation.

Summarized existing benchmarks and evaluation methods.

Abstract

Recently, tool learning with large language models (LLMs) has emerged as a promising paradigm for augmenting the capabilities of LLMs to tackle highly complex problems. Despite growing attention and rapid advancements in this field, the existing literature remains fragmented and lacks systematic organization, posing barriers to entry for newcomers. This gap motivates us to conduct a comprehensive survey of existing works on tool learning with LLMs. In this survey, we focus on reviewing existing literature from the two primary aspects (1) why tool learning is beneficial and (2) how tool learning is implemented, enabling a comprehensive understanding of tool learning with LLMs. We first explore the "why" by reviewing both the benefits of tool integration and the inherent benefits of the tool learning paradigm from six specific aspects. In terms of "how", we systematically review the literature according to a taxonomy of four key stages in the tool learning workflow: task planning, tool selection, tool calling, and response generation. Additionally, we provide a detailed summary of existing benchmarks and evaluation methods, categorizing them according to their relevance to different stages. Finally, we discuss current challenges and outline potential future directions, aiming to inspire both researchers and industrial developers to further explore this emerging and promising area. We also maintain a GitHub repository to continually keep track of the relevant papers and resources in this rising area at https://github.com/quchangle1/LLM-Tool-Survey.