SkillEvolver: Skill Learning as a Meta-Skill

TL;DR

SkillEvolver introduces an online meta-skill framework that iteratively refines domain-specific skills based on real-world failures, enabling adaptable and improved agent capabilities without retraining models.

Contribution

It presents a novel, plug-and-play meta-skill approach that refines skills through deployment feedback, outperforming static skills and previous distillation methods.

Findings

Achieves 56.8% accuracy on SkillsBench tasks, outperforming human-curated skills and baselines.

Raises mean GPU kernel speedup from 1.16 to 1.51 on KernelBench tasks.

Refinement based on deployment failures improves skill effectiveness and robustness.

Abstract

Agent skills today are static artifact: authored once -- by human curation or one-shot generation from parametric knowledge -- and then consumed unchanged, with no mechanism to improve from real use. We propose \textbf{SkillEvolver}, a lightweight, plug-and-play solution for online skill learning, in which a single meta-skill iteratively authors, deploys, and refines domain-specific skills. The learning target of SkillEvolver is the skill's prose and code, not model weights, so that the resulting artifact drops into any agent without retraining; and the meta-skill itself is just another skill, loaded through the same interface by any protocol-compliant CLI-agent. Unlike trace-distillation, the meta-skill refines only after deploying the learnt skill, such that the learning signal comes from failures another agent encounters while using it -- not from exploratory traces alone. Refinement iterations are governed by a fresh-agent overfit audit that catches possible leakage as well as deployed-skill-specific failures, including the silent-bypass mode in which a skill appears valid in content but is never invoked at runtime. On $83$ SkillsBench tasks spanning $15^{+}$ domains, SkillEvolver reaches $56.8\%$ accuracy versus $43.6\%$ for curated human skills and $29.9\%$ for the no-skill baseline; on three GPU kernel optimization tasks from KernelBench, it also raises mean speedup from $1.16$ to $1.51$ on average.