How Do Large Language Models Learn Concepts During Continual Pre-Training?

TL;DR

This paper investigates how large language models acquire, retain, and forget concepts during continual pretraining, revealing circuit-level dynamics and interactions that influence learning and interference.

Contribution

It introduces the concept circuits framework and analyzes their structure and behavior, providing new insights into concept learning in LLMs during continual training.

Findings

Concept circuits correlate with concept learning and forgetting.

Concepts show stage-wise changes during training.

Semantic similarity increases interference among concepts.

Abstract

Human beings primarily understand the world through concepts (e.g., dog), abstract mental representations that structure perception, reasoning, and learning. However, how large language models (LLMs) acquire, retain, and forget such concepts during continual pretraining remains poorly understood. In this work, we study how individual concepts are acquired and forgotten, as well as how multiple concepts interact through interference and synergy. We link these behavioral dynamics to LLMs' internal Concept Circuits, computational subgraphs associated with specific concepts, and incorporate Graph Metrics to characterize circuit structure. Our analysis reveals: (1) LLMs concept circuits provide a non-trivial, statistically significant signal of concept learning and forgetting; (2) Concept circuits exhibit a stage-wise temporal pattern during continual pretraining, with an early increase followed by gradual decrease and stabilization; (3) concepts with larger learning gains tend to exhibit greater forgetting under subsequent training; (4) semantically similar concepts induce stronger interference than weakly related ones; (5) conceptual knowledge differs in their transferability, with some significantly facilitating the learning of others. Together, our findings offer a circuit-level view of concept learning dynamics and inform the design of more interpretable and robust concept-aware training strategies for LLMs.