CoEvolve: Training LLM Agents via Agent-Data Mutual Evolution

TL;DR

CoEvolve introduces a mutual evolution framework for LLM agents and their training data, enabling adaptive, interaction-driven learning that significantly improves performance across multiple benchmarks.

Contribution

The paper presents a novel closed-loop training method where LLM agents and their data co-evolve through feedback-guided task synthesis and environment interaction.

Findings

Achieved up to 19.43% performance improvement on AppWorld.

Demonstrated consistent gains across different models and benchmarks.

Utilized feedback signals like forgetting and uncertainty for task generation.

Abstract

Reinforcement learning for LLM agents is typically conducted on a static data distribution, which fails to adapt to the agent's evolving behavior and leads to poor coverage of complex environment interactions. To address these challenges, we propose CoEvolve, an agent-data mutual evolution framework that enables LLM agents to improve through closed-loop, interaction-driven training. Specifically, CoEvolve extracts feedback signals such as forgetting and uncertainty from rollout trajectories to identify failure-prone interaction patterns, and utilizes them to guide LLM-based task synthesis. The synthesized tasks are validated through environment interaction and utilized to update the data distribution, enabling joint adaptation of the agent and its data. Extensive experiments on AppWorld and BFCL across Qwen2.5-7B, Qwen3-4B, and Qwen3-30B-A3B demonstrate consistent and significant improvements over strong base models, yielding absolute gains of 19.43%, 15.58%, and 18.14%, respectively.