How Do Large Language Models Acquire Factual Knowledge During Pretraining?

TL;DR

This paper investigates how large language models acquire and retain factual knowledge during pretraining, revealing that data quantity has limited impact, and highlighting the roles of data duplication, batch size, and knowledge dynamics.

Contribution

It provides new insights into the mechanisms of factual knowledge acquisition and retention in LLMs, including the effects of data duplication and training strategies.

Findings

More data does not significantly improve factual knowledge acquisition.

Factual knowledge retention follows a power-law decay with training steps.

Larger batch sizes improve robustness to forgetting.

Abstract

Despite the recent observation that large language models (LLMs) can store substantial factual knowledge, there is a limited understanding of the mechanisms of how they acquire factual knowledge through pretraining. This work addresses this gap by studying how LLMs acquire factual knowledge during pretraining. The findings reveal several important insights into the dynamics of factual knowledge acquisition during pretraining. First, counterintuitively, we observe that pretraining on more data shows no significant improvement in the model's capability to acquire and maintain factual knowledge. Next, there is a power-law relationship between training steps and forgetting of memorization and generalization of factual knowledge, and LLMs trained with duplicated training data exhibit faster forgetting. Third, training LLMs with larger batch sizes can enhance the models' robustness to forgetting. Overall, our observations suggest that factual knowledge acquisition in LLM pretraining occurs by progressively increasing the probability of factual knowledge presented in the pretraining data at each step. However, this increase is diluted by subsequent forgetting. Based on this interpretation, we demonstrate that we can provide plausible explanations for recently observed behaviors of LLMs, such as the poor performance of LLMs on long-tail knowledge and the benefits of deduplicating the pretraining corpus.