SkillDroid: Compile Once, Reuse Forever

TL;DR

SkillDroid is a skill-based agent that compiles LLM-guided GUI trajectories into reusable templates, significantly reducing LLM calls and improving reliability over time in mobile GUI tasks.

Contribution

It introduces a three-layer system that compiles and reuses GUI skills, enhancing efficiency and success rates compared to stateless LLM approaches.

Findings

Achieves 85.3% success rate, 23 points above baseline.

Uses 49% fewer LLM calls.

1000% success rate in replay mode across 79 rounds.

Abstract

LLM-based mobile GUI agents treat every task invocation as an independent reasoning episode, requiring a full LLM inference call at each action step. This per-step dependence makes them stateless: a task completed successfully yesterday is re-derived from scratch today, with no improvement in reliability or speed. We present SkillDroid, a three-layer skill agent that compiles successful LLM-guided GUI trajectories into parameterized skill templates (sequences of UI actions with weighted element locators and typed parameter slots) and replays them on future invocations without any LLM calls. A matching cascade (regex patterns, embedding similarity, and app filtering) routes incoming instructions to stored skills, while a failure-learning layer triggers recompilation when skill reliability degrades. Over a 150-round longitudinal evaluation with systematic instruction variation and controlled perturbations, SkillDroid achieves an 85.3% success rate (23 percentage points above a stateless LLM baseline) while using 49% fewer LLM calls. The skill replay mechanism achieves a perfect 1000% success rate across 79 replay rounds at 2.4 times the speed of full LLM execution. Most critically, the system improves with use: its success rate converges upward from 87% to 91%, while the baseline degrades from 80% to 44%.