Toolink: Linking Toolkit Creation and Using through Chain-of-Solving on Open-Source Model

TL;DR

This paper introduces Toolink, a framework that enables open-source models to create and utilize toolkits through a chain-of-solving approach, improving their task-solving abilities and generalization in real-world scenarios.

Contribution

The paper presents Toolink and LLaMA-CoS, novel methods for open-source models to generate and use toolkits via chain-of-solving, matching the performance of proprietary models like ChatGPT.

Findings

LLaMA-CoS matches ChatGPT's chain-of-solving ability on diverse tasks.

LLaMA-CoS outperforms chain-of-thought approaches in task-solving.

LLaMA-CoS generalizes well to unseen tasks and untailored toolkits.

Abstract

Large Language Models (LLMs) have demonstrated remarkable progress in utilizing tools, but their closed-source nature and high inference costs pose limitations on their adaptability, necessitating a valid method that leverages smaller, open-sourced models. In this paper, we introduce Toolink, a comprehensive framework that performs task-solving by first creating a toolkit and then integrating the planning and calling of tools through a chain-of-solving (CoS) approach. We first validate the efficacy of Toolink in harnessing the model's creativity and CoS ability on ChatGPT. Subsequently, we curate CoS-GPT, a chain-of-solving dataset designed for tool-using, and finetune the LLaMA-7B model. It results in LLaMA-CoS, a powerful open-source model with advanced tool-planning and tool-calling capabilities. Evaluation of diverse tasks from BIG-bench demonstrates its CoS ability matches that of ChatGPT while its performance surpasses the chain-of-thought approach. Further studies highlight the generalization of LLaMA-CoS to unseen tasks and showcase its capability in using toolkits not explicitly tailored for the target task, affirming its robustness in real-world scenarios.