Evolutionary Context Search for Automated Skill Acquisition

TL;DR

ECS is an evolutionary method that searches for effective context combinations to enhance large language model performance without retraining, outperforming traditional retrieval methods and enabling automated skill acquisition.

Contribution

The paper introduces ECS, a novel evolutionary approach for discovering beneficial context combinations that significantly improve language model accuracy without retraining.

Findings

ECS improves BackendBench by 27%

ECS enhances τ-bench airline by 7%

Evolved contexts transfer across different models

Abstract

Large Language Models cannot reliably acquire new knowledge post-deployment -- even when relevant text resources exist, models fail to transform them into actionable knowledge without retraining. Retrieval-Augmented Generation attempts to bridge this gap by surfacing relevant documents at inference time, yet similarity-based retrieval often fails to identify context that actually improves task performance. We introduce Evolutionary Context Search (ECS), an evolutionary method that searches context combinations using accuracy on a small development set, requiring only inference calls without weight updates. ECS moves beyond semantic similarity to discover non-obvious context pairings that significantly boost performance. Our empirical results show that ECS improves BackendBench by 27\% and $\tau$-bench airline by 7\%. The evolved contexts are model-agnostic, as those evolved with Gemini-3-Flash transfer effectively to Claude Sonnet and DeepSeek. This suggests that ECS opens a path toward automated context discovery for skill acquisition -- an efficient alternative to manual prompt engineering or costly fine-tuning.