Dynamic System Instructions and Tool Exposure for Efficient Agentic LLMs

TL;DR

The paper introduces Instruction-Tool Retrieval (ITR), a method that dynamically retrieves minimal prompts and tools for LLM agents, significantly reducing costs and errors while enabling longer, more efficient autonomous agent operations.

Contribution

ITR is a novel retrieval-based approach that dynamically composes system prompts and toolsets, improving efficiency and accuracy in long-running LLM agents.

Findings

Reduces per-step context tokens by 95%

Improves correct tool routing by 32%

Cuts episode cost by 70%

Abstract

Large Language Model (LLM) agents often run for many steps while re-ingesting long system instructions and large tool catalogs each turn. This increases cost, agent derailment probability, latency, and tool-selection errors. We propose Instruction-Tool Retrieval (ITR), a RAG variant that retrieves, per step, only the minimal system-prompt fragments and the smallest necessary subset of tools. ITR composes a dynamic runtime system prompt and exposes a narrowed toolset with confidence-gated fallbacks. Using a controlled benchmark with internally consistent numbers, ITR reduces per-step context tokens by 95%, improves correct tool routing by 32% relative, and cuts end-to-end episode cost by 70% versus a monolithic baseline. These savings enable agents to run 2-20x more loops within context limits. Savings compound with the number of agent steps, making ITR particularly valuable for long-running autonomous agents. We detail the method, evaluation protocol, ablations, and operational guidance for practical deployment.