CITI: Enhancing Tool Utilizing Ability in Large Language Models without Sacrificing General Performance

TL;DR

This paper introduces CITI, a method that improves large language models' ability to use external tools without sacrificing their overall performance, by selectively fine-tuning model components based on their importance.

Contribution

The paper proposes a novel component importance-based approach (CITI) that balances tool-utilization and general performance in LLMs through targeted fine-tuning strategies.

Findings

CITI enhances tool-utilizing ability effectively.

It maintains high general performance of LLMs.

Experimental results show superior evaluation metrics.

Abstract

Tool learning enables the Large Language Models (LLMs) to interact with the external environment by invoking tools, enriching the accuracy and capability scope of LLMs. However, previous works predominantly focus on improving model's tool-utilizing accuracy and the ability to generalize to new, unseen tools, excessively forcing LLMs to adjust specific tool-invoking pattern without considering the harm to model's general performance. This deviates from the actual applications and original intention of integrating tools to enhance model. To tackle this problem, we dissect the capability trade-offs by examining the hidden representation changes and the gradient-based importance score of model's components. Based on the analysis result, we propose a Component Importance-based Tool-utilizing ability Injection method (CITI). According to the gradient-based importance score of different components, it alleviates the capability conflicts caused by fine-tuning process by applying distinct training strategies to different components. CITI applies Mixture-Of-LoRA (MOLoRA) for important components. Meanwhile, it fine-tunes the parameters of few components deemed less important in the backbone of the LLM, while keeping other parameters frozen. CITI can effectively enhance the model's tool-utilizing capability without excessively compromising its general performance. Experimental results demonstrate that our approach achieves outstanding performance across a range of evaluation metrics.