Learning Evolving Latent Strategies for Multi-Agent Language Systems without Model Fine-Tuning

TL;DR

This paper introduces a multi-agent language system that evolves strategies over time without fine-tuning the model, using a dual-loop architecture to update latent semantic vectors through interaction and reinforcement.

Contribution

It presents a novel framework that enables continual strategy evolution in language agents without parameter fine-tuning, leveraging external latent vectors and dual-loop updates.

Findings

Latent spaces show clear convergence trajectories.

Agents develop stable, disentangled strategic styles.

Emergent ability to adapt to emotional agents without shared rewards.

Abstract

This study proposes a multi-agent language framework that enables continual strategy evolution without fine-tuning the language model's parameters. The core idea is to liberate the latent vectors of abstract concepts from traditional static semantic representations, allowing them to be continuously updated through environmental interaction and reinforcement feedback. We construct a dual-loop architecture: the behavior loop adjusts action preferences based on environmental rewards, while the language loop updates the external latent vectors by reflecting on the semantic embeddings of generated text. Together, these mechanisms allow agents to develop stable and disentangled strategic styles over long-horizon multi-round interactions. Experiments show that agents' latent spaces exhibit clear convergence trajectories under reflection-driven updates, along with structured shifts at critical moments. Moreover, the system demonstrates an emergent ability to implicitly infer and continually adapt to emotional agents, even without shared rewards. These results indicate that, without modifying model parameters, an external latent space can provide language agents with a low-cost, scalable, and interpretable form of abstract strategic representation.