Collapse of Self-trained Language Models

TL;DR

This paper investigates the effects of self-training language models on their own outputs, revealing that extended self-training causes performance degradation and output collapse, highlighting limitations of this approach.

Contribution

It provides the first systematic analysis of self-training in language models, demonstrating its practical limitations and risks of collapse.

Findings

Extended self-training degrades GPT-2 performance.

Self-training leads to repetitive, collapsed outputs.

Self-training has limited benefits and potential risks.

Abstract

In various fields of knowledge creation, including science, new ideas often build on pre-existing information. In this work, we explore this concept within the context of language models. Specifically, we explore the potential of self-training models on their own outputs, akin to how humans learn and build on their previous thoughts and actions. While this approach is intuitively appealing, our research reveals its practical limitations. We find that extended self-training of the GPT-2 model leads to a significant degradation in performance, resulting in repetitive and collapsed token output.