Semantic Context for Tool Orchestration

TL;DR

This paper introduces Semantic Context (SC) as a key component for tool orchestration, combining theoretical, empirical, and practical advances to improve adaptability and scalability in large language model-based systems.

Contribution

It provides a theoretical foundation with SC-LinUCB, empirical validation with LLMs, and a scalable pipeline (FiReAct) demonstrating effective orchestration over thousands of tools.

Findings

SC-LinUCB achieves lower regret in dynamic settings

SC improves in-context learning efficiency and robustness

SC-based retrieval enables effective large-scale tool orchestration

Abstract

This paper demonstrates that Semantic Context (SC), leveraging descriptive tool information, is a foundational component for robust tool orchestration. Our contributions are threefold. First, we provide a theoretical foundation using contextual bandits, introducing SC-LinUCB and proving it achieves lower regret and adapts favourably in dynamic action spaces. Second, we provide parallel empirical validation with Large Language Models, showing that SC is critical for successful in-context learning in both static (efficient learning) and non-stationary (robust adaptation) settings. Third, we propose the FiReAct pipeline, and demonstrate on a benchmark with over 10,000 tools that SC-based retrieval enables an LLM to effectively orchestrate over a large action space. These findings provide a comprehensive guide to building more sample-efficient, adaptive, and scalable orchestration agents.