EvoFSM: Controllable Self-Evolution for Deep Research with Finite State Machines

TL;DR

EvoFSM introduces a structured framework for self-evolution of finite state machines in AI agents, enabling adaptable, controlled problem-solving that improves accuracy and generalization across complex benchmarks.

Contribution

It proposes a novel FSM-based self-evolution method with a critic-guided refinement process and memory component, addressing instability issues in unconstrained self-optimization.

Findings

Achieves 58.0% accuracy on DeepSearch benchmark.

Demonstrates improved performance on multi-hop QA tasks.

Validates generalization in interactive decision-making scenarios.

Abstract

While LLM-based agents have shown promise for deep research, most existing approaches rely on fixed workflows that struggle to adapt to real-world, open-ended queries. Recent work therefore explores self-evolution by allowing agents to rewrite their own code or prompts to improve problem-solving ability, but unconstrained optimization often triggers instability, hallucinations, and instruction drift. We propose EvoFSM, a structured self-evolving framework that achieves both adaptability and control by evolving an explicit Finite State Machine (FSM) instead of relying on free-form rewriting. EvoFSM decouples the optimization space into macroscopic Flow (state-transition logic) and microscopic Skill (state-specific behaviors), enabling targeted improvements under clear behavioral boundaries. Guided by a critic mechanism, EvoFSM refines the FSM through a small set of constrained operations, and further incorporates a self-evolving memory that distills successful trajectories as reusable priors and failure patterns as constraints for future queries. Extensive evaluations on five multi-hop QA benchmarks demonstrate the effectiveness of EvoFSM. In particular, EvoFSM reaches 58.0% accuracy on the DeepSearch benchmark. Additional results on interactive decision-making tasks further validate its generalization.