Silent Collapse in Recursive Learning Systems

TL;DR

This paper identifies a silent collapse phenomenon in recursive learning systems where internal distributions degrade unnoticed by standard metrics, and proposes a monitoring framework for early detection and prevention.

Contribution

It uncovers the silent collapse phenomenon, characterizes its precursors, and introduces the MTR framework for early warning and adaptive regulation.

Findings

Silent collapse involves internal distribution contraction despite stable metrics.

Three trajectory precursors reliably signal impending collapse.

The MTR framework effectively prevents silent collapse without real data access.

Abstract

Recursive learning -- where models are trained on data generated by previous versions of themselves -- is increasingly common in large language models, autonomous agents, and self-supervised systems. However, standard performance metrics (loss, perplexity, accuracy) often fail to detect internal degradation before it becomes irreversible. Here we identify a phenomenon we call silent collapse: under broad recursive conditions, model internal distributions -- predictive entropy, representational diversity, and tail coverage -- progressively contract even as conventional metrics appear stable or improving. We discover that silent collapse is not abrupt. Its onset is reliably preceded by three trajectory-level precursors: (1) contraction of anchor entropy, (2) freezing of representation drift, and (3) erosion of tail coverage. These signals manifest multiple generations before any degradation in standard validation metrics, enabling early warning. Based on these precursors, we propose the MTR (Monitor--Trust--Regulator) framework, a lightweight metacognitive loop that monitors trajectory statistics, estimates a slow-timescale trust variable, and adaptively modulates the effective learning intensity. MTR provides early warning and actively prevents silent collapse without requiring access to pristine real data -- a critical advantage when original data is unavailable, contaminated, or private.