Fed-SE: Federated Self-Evolution for Privacy-Constrained Multi-Environment LLM Agents

TL;DR

Fed-SE introduces a federated self-evolution framework for privacy-preserving multi-environment LLM agents, enhancing task success rates by stabilizing gradient updates and reducing communication costs in dynamic, heterogeneous settings.

Contribution

It proposes a novel federated self-evolution approach with local parameter-efficient fine-tuning and low-rank global aggregation for LLM agents in privacy-constrained environments.

Findings

Improves average task success rates by 10% over FedIT.

Effectively stabilizes gradient updates in heterogeneous environments.

Reduces communication costs through low-rank aggregation.

Abstract

LLM agents are widely deployed in complex interactive tasks, yet privacy constraints often preclude centralized optimization and co-evolution across dynamic environments. Despite the demonstrated success of Federated Learning (FL) on static datasets, its effectiveness in open-ended, self-evolving agent systems remains largely unexplored. In such settings, the direct application of standard FL is particularly challenging, as heterogeneous tasks and sparse, trajectory-level reward signals give rise to severe gradient instability, which undermines the global optimization process. To bridge this gap, we propose Fed-SE, a Federated Self-Evolution framework for LLM agents that establishes a local evolution-global aggregation paradigm. Locally, agents employ parameter-efficient fine-tuning on filtered, high-return trajectories to achieve stable gradient updates. Globally, Fed-SE aggregates updates within a low-rank subspace, reducing communication cost across clients. Experiments across five heterogeneous environments demonstrate that Fed-SE improves average task success rates by 10\% over the state-of-the-art FedIT, validating its effectiveness in cross-environment knowledge transfer under privacy constraints.