Learning to Evolve: A Self-Improving Framework for Multi-Agent Systems via Textual Parameter Graph Optimization

TL;DR

This paper introduces TPGO, a self-improving framework for multi-agent systems that models agents and workflows as a graph, uses textual feedback for optimization, and employs meta-learning to enhance performance.

Contribution

The paper presents a novel framework combining textual gradient feedback and meta-learning to enable multi-agent systems to learn and improve their own optimization strategies.

Findings

TPGO outperforms existing methods on benchmarks like GAIA and MCP-Universe.

The framework achieves higher success rates through automated self-optimization.

GRAO effectively learns from past experiences to propose better updates.

Abstract

Designing and optimizing multi-agent systems (MAS) is a complex, labor-intensive process of "Agent Engineering." Existing automatic optimization methods, primarily focused on flat prompt tuning, lack the structural awareness to debug the intricate web of interactions in MAS. More critically, these optimizers are static; they do not learn from experience to improve their own optimization strategies. To address these gaps, we introduce Textual Parameter Graph Optimization (TPGO), a framework that enables a multi-agent system to learn to evolve. TPGO first models the MAS as a Textual Parameter Graph (TPG), where agents, tools, and workflows are modular, optimizable nodes. To guide evolution, we derive "textual gradients," structured natural language feedback from execution traces, to pinpoint failures and suggest granular modifications. The core of our framework is Group Relative Agent Optimization (GRAO), a novel meta-learning strategy that learns from historical optimization experiences. By analyzing past successes and failures, GRAO becomes progressively better at proposing effective updates, allowing the system to learn how to optimize itself. Extensive experiments on complex benchmarks like GAIA and MCP-Universe show that TPGO significantly enhances the performance of state-of-the-art agent frameworks, achieving higher success rates through automated, self-improving optimization.