FlashEvolve: Accelerating Agent Self-Evolution with Asynchronous Stage Orchestration

TL;DR

FlashEvolve introduces an asynchronous framework for LLM-based agent evolution, significantly reducing wall-clock time by overlapping stages and managing data staleness through version tracking and repair policies.

Contribution

It replaces synchronized stage execution with asynchronous workers, enabling overlapping and improving throughput in agent evolution workflows.

Findings

Proposal throughput increased by 3.5x on local vLLM

Proposal throughput increased by 4.9x on API serving

Effective handling of language-space staleness improves efficiency

Abstract

LLM-based evolution has emerged as a promising way to improve agents by refining non-parametric artifacts, but its wall-clock cost remains a major bottleneck. We identify that this cost comes from synchronized stage execution and imbalance inside each LLM-heavy stage. We present FlashEvolve, an efficient framework that replaces synchronized execution with asynchronous workers and queues, allowing different stages and steps to overlap. To handle data staleness introduced by asynchrony, FlashEvolve tracks artifact versions and applies different policies to update, discard, or patch stale artifacts. Unlike weight-space staleness in asynchronous RL, language-space staleness is inspectable and repairable: a stale artifact is not just delayed work, but readable evidence that the LLM can reflect on, revise, and turn into useful evolution signal. FlashEvolve further improves throughput and token efficiency with speculative stage completion and adaptive workflow control. On GEPA workloads, FlashEvolve improves proposal throughput by $3.5\times$ on local vLLM and $4.9\times$ on API serving over synchronous GEPA. The same design also applies to ACE and Meta-Harness.