Continual Fine-Tuning of Large Language Models via Program Memory

TL;DR

ProCL introduces a program memory-based continual LoRA framework for LLMs, enabling rapid adaptation and knowledge retention without extra inference cost, inspired by neuroscience.

Contribution

It proposes a novel structured program memory system for LoRA adapters, improving continual learning by balancing plasticity and stability.

Findings

Enhanced retention and reduced forgetting in benchmarks.

Operates within LoRA without additional inference overhead.

Organizes adapters into dynamic, input-conditioned memory slots.

Abstract

Parameter-Efficient Fine-Tuning (PEFT), particularly Low-Rank Adaptation (LoRA), has become a standard approach for adapting Large Language Models (LLMs) under limited compute. However, in continual settings where models are updated sequentially with small datasets, conventional LoRA updates struggle to balance rapid adaptation and knowledge retention. Existing methods typically treat the low-rank space as a homogeneous update region, lacking mechanisms to regulate how short-term updates are consolidated over time. We propose a continual LoRA framework with \textbf{Pro}gram memory, inspired by \textbf{C}omplementary \textbf{L}earning Systems in neuroscience. Our approach, dubbed \textbf{ProCL}, organizes LoRA adapters into structured program memory slots that are dynamically retrieved through input-conditioned attention. This enables rapid and localized adaptation, encouraging similar inputs to reuse shared adapter regions while reserving unused capacity for future data. The slots are then combined with the underlying adapter, which maintains a distributed representation that gradually accumulates knowledge across tasks to balance plasticity and stability. Our method operates entirely within the LoRA parameterization and incurs no additional inference cost. Experiments on diverse benchmarks demonstrate improved retention and reduced catastrophic forgetting over other continual LoRA strategies.