Mitigating the Learning Bias towards Repetition by Self-Contrastive Training for Open-Ended Generation

TL;DR

This paper introduces self-contrastive training to reduce repetition in language models, addressing their bias towards simple repetitive patterns and improving open-ended text generation quality.

Contribution

It proposes a novel self-contrastive training method that penalizes early model predictions of repetition, effectively mitigating repetition bias in language models.

Findings

Reduces repetitive outputs in language models

Maintains fluency while decreasing repetition

Identifies longer-range dependencies as a cause of repetition loops

Abstract

Despite the huge progress in myriad generation tasks, pretrained language models (LMs) such as GPT2 still tend to generate repetitive texts with maximization-based decoding algorithms for open-ended generation. We attribute their overestimation of token-level repetition probabilities to the learning bias: LMs capture simple repetitive patterns faster with the MLE loss. We propose self-contrastive training to penalize the output of a premature checkpoint of the same model when it incorrectly predicts repetition, which is shown to mitigate repetition effectively while maintaining fluency on two datasets. Furthermore, we find that LMs use longer-range dependencies to predict repetitive tokens than non-repetitive ones, which may be the cause of sentence-level repetition loops.