Dynamic Mixture of Latent Memories for Self-Evolving Agents

TL;DR

The paper introduces MoLEM, a dynamic mixture-of-experts framework for self-evolving agents that continually acquire knowledge without forgetting, demonstrated across math, science, and code tasks.

Contribution

MoLEM is a novel generative mixture of latent memory approach that enables continual learning without catastrophic forgetting by internalizing knowledge into additional modules.

Findings

MoLEM improves average accuracy by 10.40% over baseline.

It effectively preserves knowledge across multiple training stages.

Competing methods do not consistently outperform the baseline.

Abstract

Achieving self-evolution in intelligent agents requires the continual accumulation of new knowledge across changing task sequences without forgetting previously acquired abilities. Existing approaches either internalize knowledge by updating model parameters, which induces catastrophic forgetting, or rely on external memory, which fails to genuinely enhance the model's intrinsic capabilities. We propose MoLEM, a generative mixture of latent memory framework based on a dynamic mixture-of-experts (MoE). We treat multiple experts as independent carriers to generate memory. A router selects and weights experts through key-query matching, and the aggregated latent memory is injected into the reasoning process. The base model for reasoning remains entirely frozen, with all experiential knowledge internalized into the additional modules, avoiding catastrophic forgetting. For continual learning, each training stage is paired with a lightweight autoencoder that selects the appropriate routing group at inference, and inputs that match no stage fall back to the pretrained model. Experiments train the framework on continual-learning sequences spanning math, science, and code domains. After training, we evaluate the framework on the corresponding test sets to measure task learning and competence preservation across continual adaptation stages. After the full continual-learning sequence, our method improves the average accuracy by 10.40% over the Vanilla pretrained baseline, while none of the competing methods consistently exceed this baseline across different training orders.