Verification-Guided Context Optimization for Tool Calling via Hierarchical LLMs-as-Editors

TL;DR

This paper introduces VGCO, a framework that uses hierarchical LLM-based editors to automatically refine tool documentation and context, significantly improving tool use accuracy and robustness in large language models.

Contribution

VGCO is a novel framework employing hierarchical, verification-guided LLM editors for automatic context optimization in tool calling, addressing scalability and ambiguity issues.

Findings

Improves tool calling accuracy and robustness.

Enables cost-efficient, targeted context editing.

Achieves better generalization across LLMs.

Abstract

Tool calling enables large language models (LLMs) to interact with external environments through tool invocation, providing a practical way to overcome the limitations of pretraining. However, the effectiveness of tool use depends heavily on the quality of the associated documentation and knowledge base context. These materials are usually written for human users and are often misaligned with how LLMs interpret information. This problem is even more pronounced in industrial settings, where hundreds of tools with overlapping functionality create challenges in scalability, variability, and ambiguity. We propose Verification-Guided Context Optimization (VGCO), a framework that uses LLMs as editors to automatically refine tool-related documentation and knowledge base context. VGCO works in two stages. First, Evaluation collects real-world failure cases and identifies mismatches between tools and their context. Second, Optimization performs hierarchical editing through offline learning with structure-aware, in-context optimization. The novelty of our LLM editors has three main aspects. First, they use a hierarchical structure that naturally integrates into the tool-calling workflow. Second, they are state-aware, action-specific, and verification-guided, which constrains the search space and enables efficient, targeted improvements. Third, they enable cost-efficient sub-task specialization, either by prompt engineering large editor models or by post-training smaller editor models. Unlike prior work that emphasizes multi-turn reasoning, VGCO focuses on the single-turn, large-scale tool-calling problem and achieves significant improvements in accuracy, robustness, and generalization across LLMs.