Scaling Laws for Forgetting When Fine-Tuning Large Language Models

TL;DR

This paper investigates the extent and scaling behavior of forgetting in large language models during fine-tuning, revealing a predictable inverse relationship and highlighting challenges for safety and knowledge retention.

Contribution

It provides the first precise scaling laws for forgetting in fine-tuning LLMs, especially with parameter-efficient methods like LoRA, and shows forgetting cannot be mitigated by early stopping.

Findings

Forgetting scales as a shifted power law with parameters and steps.

Parameter-efficient fine-tuning still suffers from catastrophic forgetting.

Forgetting impacts knowledge, reasoning, and safety features.

Abstract

We study and quantify the problem of forgetting when fine-tuning pre-trained large language models (LLMs) on a downstream task. We find that parameter-efficient fine-tuning (PEFT) strategies, such as Low-Rank Adapters (LoRA), still suffer from catastrophic forgetting. In particular, we identify a strong inverse linear relationship between the fine-tuning performance and the amount of forgetting when fine-tuning LLMs with LoRA. We further obtain precise scaling laws that show forgetting increases as a shifted power law in the number of parameters fine-tuned and the number of update steps. We also examine the impact of forgetting on knowledge, reasoning, and the safety guardrails trained into Llama 2 7B chat. Our study suggests that forgetting cannot be avoided through early stopping or by varying the number of parameters fine-tuned. We believe this opens up an important safety-critical direction for future research to evaluate and develop fine-tuning schemes which mitigate forgetting