How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse

TL;DR

This paper provides a statistical analysis of model collapse in language models trained on synthetic data, showing it cannot be avoided with synthetic data alone but can be mitigated by mixing real data.

Contribution

It introduces a statistical model to analyze language model collapse and estimates the threshold of synthetic data mixing needed to prevent it.

Findings

Model collapse cannot be avoided when training only on synthetic data.

Mixing real and synthetic data below a certain threshold prevents collapse.

Empirical results support the theoretical analysis.

Abstract

The phenomenon of model collapse, introduced in (Shumailov et al., 2023), refers to the deterioration in performance that occurs when new models are trained on synthetic data generated from previously trained models. This recursive training loop makes the tails of the original distribution disappear, thereby making future-generation models forget about the initial (real) distribution. With the aim of rigorously understanding model collapse in language models, we consider in this paper a statistical model that allows us to characterize the impact of various recursive training scenarios. Specifically, we demonstrate that model collapse cannot be avoided when training solely on synthetic data. However, when mixing both real and synthetic data, we provide an estimate of a maximal amount of synthetic data below which model collapse can eventually be avoided. Our theoretical conclusions are further supported by empirical validations.