Evolve: A Persistent Knowledge Lifecycle for Small Language Models

TL;DR

Evolve enhances small language models by integrating a persistent, semantically organized knowledge store, significantly improving accuracy and efficiency through offline consolidation and reuse of knowledge sections.

Contribution

The paper introduces a novel knowledge lifecycle architecture that combines a small local model with a teacher-compiled, consolidatable knowledge store for improved performance.

Findings

Accuracy improved from 20-33% to 60-84% with Evolve.

Teacher invocations reduced by over 50% through knowledge reuse.

Knowledge store compression of 31-33.5% while maintaining accuracy.

Abstract

Evolve pairs a small local language model with a persistent, teacher-compiled knowledge store -- refined through sleep consolidation and usage-driven refresh -- to deliver substantial accuracy gains over the model's parametric baseline while amortizing teacher costs through cross-query knowledge reuse. Rather than retrieving document fragments at query time, Evolve constructs a store of semantically coherent sections compiled by teacher models at natural conceptual boundaries; new sections are staged on acquisition, consolidated offline through teacher-mediated merging, and refreshed inline when expired. A 2B-parameter local model handles classification and generation; large teacher models are invoked only for knowledge operations. Across 750 benchmark queries spanning custom specialist questions, NaturalQuestions, and TriviaQA, the 2B model augmented by Evolve improves from 20-33% baseline accuracy to 60-84% (+40-52pp) while reducing teacher invocations by over 50% through reuse. Post-consolidation compresses the knowledge store by 31-33.5% across three independent benchmarks while preserving accuracy; section-based retrieval outperforms chunk-based retrieval by 5-9pp across every lifecycle condition. The architecture supports two generation modes over the same lifecycle -- suppress (strict section-only grounding, auditable) and augment (section-supplemented responses).