SkillMOO: Multi-Objective Optimization of Agent Skills for Software Engineering

TL;DR

SkillMOO introduces a multi-objective framework that optimizes agent skill bundles for software engineering tasks by balancing success rates and inference costs, outperforming static approaches.

Contribution

It presents a novel cost-aware, multi-objective search method for evolving skill bundles using LLMs and Pareto optimization, improving over traditional static skill configurations.

Findings

SkillMOO achieves top pass rate on 11 of 12 tasks.

Cost reductions of up to 31.7% over static skill bundles.

Pruning and substitution are key successful skill edits.

Abstract

Agent skills are increasingly used to configure coding agents for software engineering (SE) tasks, yet current practice treats them as static, hand-crafted assets, or evolved on pass rate alone. This is insufficient: a skill can improve task success while substantially raising token cost, or introducing misleading guidance. We argue that SE agent skill bundles can be treated as multi-objective search objects and present SkillMOO, a framework that evolves skill bundles through LLM-proposed edits and NSGA-II Pareto selection on pass rate and inference cost. Evaluated across all 16 SkillsBench SE tasks, SkillMOO achieves the top pass rate rank on 11 of 12 non-zero-pass tasks while achieving cost reductions of up to 31.7% over static bundles, with pass rate gains up to 21 percentage points. Analysis of 38 skill edits shows that pruning and substitution dominate successful operations, offering actionable principles for skill bundle design. Thereby, the current practice of deploying skills without cost-aware validation leaves better skill configurations unexplored, motivating a new class of cost-aware, search-based skill engineering.