Decocted Experience Improves Test-Time Inference in LLM Agents

TL;DR

This paper introduces 'decocted experience' as a method to enhance test-time inference in LLM agents by constructing better context from accumulated experience, improving reasoning and task performance.

Contribution

It systematically analyzes how to derive and utilize experience to construct effective context, highlighting decocted experience as a key mechanism for performance improvement.

Findings

Decocted experience helps organize and extract salient information from past interactions.

Effective context construction from experience improves reasoning and agentic task performance.

The approach is validated across math reasoning, web browsing, and software engineering tasks.

Abstract

There is growing interest in improving LLMs without updating model parameters. One well-established direction is test-time scaling, where increased inference-time computation (e.g., longer reasoning, sampling, or search) is used to improve performance. However, for complex reasoning and agentic tasks, naively scaling test-time compute can substantially increase cost and still lead to wasted budget on suboptimal exploration. In this paper, we explore \emph{context} as a complementary scaling axis for improving LLM performance, and systematically study how to construct better inputs that guide reasoning through \emph{experience}. We show that effective context construction critically depends on \emph{decocted experience}. We present a detailed analysis of experience-augmented agents, studying how to derive context from experience, how performance scales with accumulated experience, what characterizes good context, and which data structures best support context construction. We identify \emph{decocted experience} as a key mechanism for effective context construction: extracting essence from experience, organizing it coherently, and retrieving salient information to build effective context. We validate our findings across reasoning and agentic tasks, including math reasoning, web browsing, and software engineering.