FLEX: Continuous Agent Evolution via Forward Learning from Experience

TL;DR

FLEX introduces a gradient-free method for LLM agents to continuously evolve through experience, significantly improving performance in reasoning and prediction tasks by constructing an experience library and enabling inheritance.

Contribution

The paper presents FLEX, a novel gradient-free framework for continuous agent evolution through experience, including inheritance and scalability, advancing beyond static LLM agents.

Findings

Up to 23% improvement on AIME25

Up to 10% improvement on USPTO50k

Up to 14% improvement on ProteinGym

Abstract

Autonomous agents driven by Large Language Models (LLMs) have revolutionized reasoning and problem-solving but remain static after training, unable to grow with experience as intelligent beings do during deployment. We introduce Forward Learning with EXperience (FLEX), a gradient-free learning paradigm that enables LLM agents to continuously evolve through accumulated experience. Specifically, FLEX cultivates scalable and inheritable evolution by constructing a structured experience library through continual reflection on successes and failures during interaction with the environment. FLEX delivers substantial improvements on mathematical reasoning, chemical retrosynthesis, and protein fitness prediction (up to 23% on AIME25, 10% on USPTO50k, and 14% on ProteinGym). We further identify a clear scaling law of experiential growth and the phenomenon of experience inheritance across agents, marking a step toward scalable and inheritable continuous agent evolution. Project Page: https://flex-gensi-thuair.github.io.