What Does Loss Optimization Actually Teach, If Anything? Knowledge Dynamics in Continual Pre-training of LLMs

TL;DR

This paper investigates how continual pre-training of large language models affects factual knowledge and skills, revealing that loss reduction does not reliably indicate learning progress or knowledge consolidation.

Contribution

It introduces a controlled benchmark and diagnostic probes to measure knowledge dynamics during CPT, highlighting the misalignment between loss and actual learning.

Findings

Factual learning is unstable and non-monotonic during CPT.

Knowledge pathways reconfigure rapidly, causing forgetting and narrow acquisition windows.

Loss decreases monotonically, but learning progress and knowledge consolidation are inconsistent.

Abstract

Continual Pre-Training (CPT) is widely used for acquiring and updating factual knowledge in LLMs. This practice treats loss as a proxy for knowledge learning, while offering no grounding into how it changes during training. We study CPT as a knowledge learning process rather than a solely optimization problem. We construct a controlled, distribution-matched benchmark of factual documents and interleave diagnostic probes directly into the CPT loop, enabling epoch-level measurement of knowledge acquisition dynamics and changes in Out-Of-Domain (OOD) general skills (e.g., math). We further analyze how CPT reshapes knowledge circuits during training. Across three instruction-tuned LLMs and multiple CPT strategies, optimization and learning systematically diverge as loss decreases monotonically while factual learning is unstable and non-monotonic. Acquired facts are rarely consolidated, learning is strongly conditioned on prior exposure, and OOD performance degrades from early epochs. Circuit analysis reveals rapid reconfiguration of knowledge pathways across epochs, providing an explanation for narrow acquisition windows and systematic forgetting. These results show that loss optimization is misaligned with learning progress in CPT and motivate evaluation of stopping criteria based on task-level learning dynamics.