Self-Consolidation for Self-Evolving Agents

TL;DR

This paper introduces a self-evolving framework for LLM agents that leverages contrastive reflection and self-consolidation to improve lifelong learning by utilizing both successes and failures.

Contribution

It presents a novel self-consolidation mechanism that distills textual experiences into learnable parameters, enhancing agent evolution beyond traditional retrieval methods.

Findings

Improved long-term evolution of LLM agents.

Effective summarization of error patterns.

Enhanced internalization of experience.

Abstract

While large language model (LLM) agents have demonstrated impressive problem-solving capabilities, they typically operate as static systems, lacking the ability to evolve through lifelong interaction. Existing attempts to bridge this gap primarily rely on retrieving successful past trajectories as demonstrations. However, this paradigm faces two critical limitations. First, by focusing solely on success, agents overlook the rich pedagogical value embedded in failed attempts, preventing them from identifying and avoiding recurrent pitfalls. Second, continually accumulating textual experiences not only increases the time consumption during retrieval but also inevitably introduces noise and exhausts the largest context window of current LLMs. To address these challenges, we propose a novel self-evolving framework for LLM agents that introduces a complementary evolution mechanism: First, a contrastive reflection strategy is introduced to explicitly summarize error-prone patterns and capture reusable insights. Second, we propose a self-consolidation mechanism that distills non-parametric textual experience into compact learnable parameters. This enables the agent to internalize extensive historical experience directly into its latent space. Extensive experiments demonstrate the advantages of our method in long-term agent evolution.