TUMS: Enhancing Tool-use Abilities of LLMs with Multi-structure Handlers

TL;DR

TUMS is a framework that improves LLMs' tool-use abilities by transforming tool interaction into parameter-level processing, utilizing multi-structure handlers to enhance accuracy and efficiency in task execution.

Contribution

The paper introduces TUMS, a novel framework with multi-structure handlers that significantly enhances LLMs' tool-use capabilities through task decomposition and parameter generation.

Findings

19.6% improvement on easy benchmarks

50.6% improvement on hard benchmarks

Key components validated by ablation experiments

Abstract

Recently, large language models(LLMs) have played an increasingly important role in solving a wide range of NLP tasks, leveraging their capabilities of natural language understanding and generating. Integration with external tools further enhances LLMs' effectiveness, providing more precise, timely, and specialized responses. However, LLMs still encounter difficulties with non-executable actions and improper actions, which are primarily attributed to incorrect parameters. The process of generating parameters by LLMs is confined to the tool level, employing the coarse-grained strategy without considering the different difficulties of various tools. To address this issue, we propose TUMS, a novel framework designed to enhance the tool-use capabilities of LLMs by transforming tool-level processing into parameter-level processing. Specifically, our framework consists of four key components: (1) an intent recognizer that identifies the user's intent to help LLMs better understand the task; (2) a task decomposer that breaks down complex tasks into simpler subtasks, each involving a tool call; (3) a subtask processor equipped with multi-structure handlers to generate accurate parameters; and (4) an executor. Our empirical studies have evidenced the effectiveness and efficiency of the TUMS framework with an average of 19.6\% and 50.6\% improvement separately on easy and hard benchmarks of ToolQA, meanwhile, we demonstrated the key contribution of each part with ablation experiments, offering more insights and stimulating future research on Tool-augmented LLMs.