Self-Consolidating Language Models: Continual Knowledge Incorporation from Context

TL;DR

This paper introduces SCoL, a framework enabling language models to continually update their knowledge by selectively consolidating information into specific model layers using meta-reinforcement learning.

Contribution

SCoL is a novel post-training method that trains LLMs to generate textual update instructions for selective layer consolidation, improving knowledge retention and scalability.

Findings

SCoL outperforms prompting, summarization, and finetuning baselines in knowledge acquisition.

Learned update patterns align with high Fisher information layers, indicating targeted plasticity.

SCoL transfers effectively from shorter to longer context streams, demonstrating scalability.

Abstract

Large language models (LLMs) increasingly receive information as streams of passages, conversations, and long-context workflows. While longer context windows expose more evidence, they do not ensure that useful information is preserved and reused. We study continual context consolidation: writing current context into model weights while limiting interference with previously consolidated information. We propose \textbf{S}elf-\textbf{Co}nsolidating \textbf{L}anguage Models (SCoL), a post-training framework in which, given current context, an LLM learns to generate textual update instructions specifying which of its own Transformer layers should be updated. Because committed updates change the model that later generates future selections, we train SCoL with meta-reinforcement learning over an evolving model state. We instantiate SCoL with supervised QA rewards on SQuAD knowledge incorporation and intrinsic likelihood-based rewards for LongBench v2 long-context consolidation. Across both settings, SCoL improves acquisition and retention over prompting, summarization, batch test-time training, and sequential finetuning baselines. Analysis of learned selection patterns shows that SCoL encourages the LLM to generate sparse update locations that align with layers of high Fisher information, suggesting that the model learns to route plasticity toward loss-sensitive regions while limiting interference. Moreover, SCoL transfers from shorter meta-training streams to longer LongBench v2 streams at evaluation, suggesting that our framework supports scalable streaming consolidation.