SkillRAE: Agent Skill-Based Context Compilation for Retrieval-Augmented Execution

TL;DR

SkillRAE introduces a two-stage retrieval-augmented execution method that constructs a compact, grounded, and task-specific skill context for LLM-based agents, significantly improving performance.

Contribution

It proposes a novel skill-based context compilation approach with offline indexing and online retrieval, enhancing retrieval-augmented execution for artifact-rich tasks.

Findings

Achieves 11.7% improvement over SOTA on SkillsBench.

Effective context compilation is crucial, beyond prompt addition.

Constructs a multi-level skill graph for better organization.

Abstract

Large Language Model (LLM)-based agents (e.g., OpenClaw) increasingly rely on reusable skill libraries to solve artifact-rich tasks such as document-centric workflows and data-intensive analysis. As these libraries grow, a few works have attempted to study the Retrieval-Augmented Execution (RAE), which often first retrieves some external skills and other knowledge, then compiles the context using retrieved skills, and finally executes the task. Existing works mainly focus on optimizing skill retrieval and task execution, and they pay little attention to how to effectively organize the selected skill evidence in a form that is compact, grounded, and immediately usable for the downstream executors to complete tasks. To fill this gap, we propose SkillRAE, a two-stage RAE approach focusing on skill-based context compilation, which consists of the offline and online stages. Specifically, in the offline indexing stage, it builds a multi-level skill graph over skill communities, skills, and reusable subunits, for capturing their relationships. In the online retrieval stage, it first performs skill-ranked retrieval with selected-subunit evidence export in the graph, and then applies rescue-aware compact compilation to recover the key evidence. Together, these components compile a coarse-ranked skill set into a task-specific context that is compact, grounded, and immediately usable. Experiments on two public benchmarks show that SkillRAE achieves a significant improvement over baselines for RAE. For example, on SkillsBench, it achieves an improvement of 11.7% over the SOTA method. Ablation studies further show that our context compilation is crucial, instead of a mere prompt addition.