ProTIP: Progressive Tool Retrieval Improves Planning

TL;DR

ProTIP is a contrastive learning framework that enhances multi-step planning with dynamic tool retrieval, outperforming existing methods by improving recall and tool accuracy without explicit subtask labels.

Contribution

ProTIP introduces a novel contrastive learning approach for progressive tool retrieval that implicitly handles task decomposition and maintains subtask-tool atomicity.

Findings

ProTIP achieves 24% higher Recall@10 for tool retrieval.

ProTIP improves tool accuracy by 41% in plan generation.

Outperforms ChatGPT-based task decomposition on ToolBench dataset.

Abstract

Large language models (LLMs) are increasingly employed for complex multi-step planning tasks, where the tool retrieval (TR) step is crucial for achieving successful outcomes. Two prevalent approaches for TR are single-step retrieval, which utilizes the complete query, and sequential retrieval using task decomposition (TD), where a full query is segmented into discrete atomic subtasks. While single-step retrieval lacks the flexibility to handle "inter-tool dependency," the TD approach necessitates maintaining "subtask-tool atomicity alignment," as the toolbox can evolve dynamically. To address these limitations, we introduce the Progressive Tool retrieval to Improve Planning (ProTIP) framework. ProTIP is a lightweight, contrastive learning-based framework that implicitly performs TD without the explicit requirement of subtask labels, while simultaneously maintaining subtask-tool atomicity. On the ToolBench dataset, ProTIP outperforms the ChatGPT task decomposition-based approach by a remarkable margin, achieving a 24% improvement in Recall@K=10 for TR and a 41% enhancement in tool accuracy for plan generation.