A theoretical basis for model collapse in recursive training

TL;DR

This paper provides a theoretical explanation for model collapse in recursive training of generative models, showing how external sources influence the asymptotic behavior of the learned distribution.

Contribution

It introduces a theoretical framework distinguishing two asymptotic regimes based on the presence of external sampling sources during recursive training.

Findings

Recursive training can lead to different asymptotic behaviors.

External sources, even minor, significantly affect model convergence.

Theoretical insights explain collapse phenomena in generative models.

Abstract

It is known that recursive training from generative models can lead to the so called `collapse' of the simulated probability distribution. This note shows that one in fact gets two different asymptotic behaviours depending on whether an external source, howsoever minor, is also contributing samples.