From History to State: Constant-Context Skill Learning for LLM Agents

TL;DR

This paper introduces a constant-context skill learning framework for LLM agents that improves efficiency and privacy by moving procedural context from prompts into learned weights, enabling better performance across multiple tasks.

Contribution

The authors propose a novel context-to-weights framework that learns reusable procedures in lightweight modules, reducing prompt size and enhancing privacy and performance for LLM agents.

Findings

Achieves high success rates on ALFWorld, WebShop, and SciWorld benchmarks.

Reduces prompt tokens per turn by 2-7 times compared to ReAct baselines.

Matches or exceeds published agent-training results.

Abstract

Large language model (LLM) agents are increasingly used to operate browsers, files, code and tools, making personal assistants a natural deployment target. Yet personal agents face a privacy-cost-capability tension: cloud models execute multi-step workflows well but expose sensitive intermediate context to external APIs, while local models preserve privacy but remain less reliable. Both settings also pay repeatedly for long skill prompts and growing histories. We propose constant-context skill learning, a context-to-weights framework for recurring agent workflows: reusable procedures are learned in lightweight task-family modules, while inference conditions only on the current observation and a compact state block. A deterministic tracker renders this state block from task progress and supplies aligned subgoal rewards, so each module can be trained with step-level SFT and refined through online RL. Across ALFWorld, WebShop, and SciWorld, our agents achieve strong performance across Qwen3-4B, Qwen3-8B and Llama-3.1-8B. With Qwen3-8B, SFT+RL reaches 89.6\% unseen success on ALFWorld, 76.8\% success on WebShop, and 66.4\% unseen success on SciWorld. They match or exceed strong published agent-training results while reducing prompt tokens per turn by 2--7$\times$ relative to controlled ReAct prompting baselines, showing that procedural context can be moved from prompts into weights.