AgentPSO: Evolving Agent Reasoning Skill via Multi-agent Particle Swarm Optimization

TL;DR

AgentPSO introduces a particle-swarm-inspired framework that evolves multi-agent reasoning skills in natural language, enhancing problem-solving abilities without modifying the underlying language model.

Contribution

It presents a novel method for evolving reasoning skills in multi-agent systems using a particle-swarm approach, improving performance and transferability.

Findings

AgentPSO outperforms static single-agent and test-time multi-agent baselines.

Evolved skills transfer across different benchmarks and models.

The framework captures reusable reasoning procedures.

Abstract

Multi-agent reasoning has shown promise for improving the problem-solving ability of large language models by allowing multiple agents to explore diverse reasoning paths. However, most existing multi-agent methods rely on inference-time debate or aggregation, which can be vulnerable to incorrect peer influence and biased consensus. Moreover, the agents themselves remain static, as their underlying reasoning skills do not evolve across tasks. In this paper, we introduce AgentPSO, a particle-swarm-inspired framework for evolving multi-agent reasoning skills. AgentPSO treats each agent as a particle-like reasoner whose state is a natural-language skill and whose velocity is a semantic update direction, iteratively moving agents toward stronger skill states to improve both individual and collective reasoning performance. Across training iterations, each agent updates its skill by combining its previous velocity, personal-best skill, global-best skill, and a self-reflective direction derived from peer reasoning trajectories. This enables agents to learn reusable reasoning behaviors from both their own experiences and the strongest skills discovered by the population, without updating the parameters of the backbone language model. Experiments on mathematical and general reasoning benchmarks show that AgentPSO improves over static single-agent skills and test-time-only multi-agent reasoning baselines. The evolved skills further transfer across benchmarks and to another backbone model, suggesting that AgentPSO captures reusable reasoning procedures rather than merely optimizing benchmark-specific prompts. Code is open-sourced at https://github.com/HYUNMIN-HWANG/AgentPSO/.