A Survey on Self-Evolution of Large Language Models

TL;DR

This survey reviews self-evolution methods for large language models, highlighting a new iterative framework that enables models to autonomously learn and improve from their own generated experiences, aiming for scalable superintelligence.

Contribution

It introduces a comprehensive conceptual framework and taxonomy for self-evolution in LLMs, categorizing objectives and outlining future research directions.

Findings

Categorization of self-evolution objectives

A four-phase iterative framework for self-evolution

Identification of key challenges and future directions

Abstract

Large language models (LLMs) have significantly advanced in various fields and intelligent agent applications. However, current LLMs that learn from human or external model supervision are costly and may face performance ceilings as task complexity and diversity increase. To address this issue, self-evolution approaches that enable LLM to autonomously acquire, refine, and learn from experiences generated by the model itself are rapidly growing. This new training paradigm inspired by the human experiential learning process offers the potential to scale LLMs towards superintelligence. In this work, we present a comprehensive survey of self-evolution approaches in LLMs. We first propose a conceptual framework for self-evolution and outline the evolving process as iterative cycles composed of four phases: experience acquisition, experience refinement, updating, and evaluation. Second, we categorize the evolution objectives of LLMs and LLM-based agents; then, we summarize the literature and provide taxonomy and insights for each module. Lastly, we pinpoint existing challenges and propose future directions to improve self-evolution frameworks, equipping researchers with critical insights to fast-track the development of self-evolving LLMs. Our corresponding GitHub repository is available at https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/Awesome-Self-Evolution-of-LLM