SkillGraph: Graph Foundation Priors for LLM Agent Tool Sequence Recommendation

TL;DR

This paper introduces SkillGraph, a graph-based prior derived from successful LLM agent tool sequences, to improve tool ordering accuracy by capturing workflow dependencies beyond semantic similarity.

Contribution

The paper presents SkillGraph, a novel graph foundation prior and a two-stage framework that significantly enhances tool sequence recommendation for LLM agents.

Findings

SkillGraph improves Kendall-$ au$ from -0.433 to +0.613 on API-Bank.

The two-stage framework outperforms LLaMA-3.1-8B rerankers.

Method achieves Set-F1 = 0.271 and Kendall-$ au$ = 0.096 on ToolBench.

Abstract

LLM agents must select tools from large API libraries and order them correctly. Existing methods use semantic similarity for both retrieval and ordering, but ordering depends on inter-tool data dependencies that are absent from tool descriptions. As a result, semantic-only methods can produce negative Kendall-$\tau$ in structured workflow domains. We introduce SkillGraph, a directed weighted execution-transition graph mined from 49,831 successful LLM agent trajectories, which encodes workflow-precedence regularities as a reusable graph foundation prior. Building on this graph foundation prior, we propose a two-stage decoupled framework: GS-Hybrid retrieval for candidate selection and a learned pairwise reranker for ordering. On ToolBench (9,965 test instances; ~16,000 tools), the method reaches Set-F1 = 0.271 and Kendall-$\tau$ = 0.096; on API-Bank, Kendall-$\tau$ improves from -0.433 to +0.613. Under identical Stage-1 inputs, the learned reranker also outperforms LLaMA-3.1-8B Stage-2 rerankers.