Continual Knowledge Updating in LLM Systems: Learning Through Multi-Timescale Memory Dynamics

TL;DR

This paper proposes Memini, an associative memory model inspired by biological multi-timescale dynamics, enabling continual knowledge updating in LLMs through self-organizing external memory.

Contribution

It introduces a novel external memory system with coupled fast and slow variables, allowing dynamic knowledge reorganization without explicit management.

Findings

Memory reorganizes through its own dynamics, enabling continual learning.

Coupled variables facilitate immediate association and gradual consolidation.

The model mimics biological synaptic consolidation processes.

Abstract

LLMs are trained once, then deployed into a world that never stops changing. External memory compensates for this, but most systems manage it explicitly rather than letting it adapt on its own. Biological memory works differently: coupled multi-timescale dynamics make new associations immediately usable, strengthen what repetition confirms, and let the rest fade. We argue that external memory should follow a similar principle. In Memini, this view takes the form of an associative memory that organizes knowledge as a directed graph. Each edge carries two coupled internal variables, one fast and one slow, following the Benna-Fusi model of synaptic consolidation. From this coupling, episodic sensitivity, gradual consolidation, and selective forgetting emerge as facets of a single mechanism, reframing external memory as a learning substrate that reorganizes through its own dynamics.