RankGen: Improving Text Generation with Large Ranking Models

TL;DR

RankGen is a large ranking model that improves text generation quality by scoring and selecting more relevant, coherent outputs from pretrained language models, outperforming existing decoding methods.

Contribution

Introduces RankGen, a 1.2B parameter ranking model trained with contrastive learning to enhance text generation quality across various language models.

Findings

RankGen outperforms traditional decoding algorithms in automatic metrics.

Human evaluations favor RankGen's relevance and coherence.

RankGen improves continuity and relevance in generated text.

Abstract

Given an input sequence (or prefix), modern language models often assign high probabilities to output sequences that are repetitive, incoherent, or irrelevant to the prefix; as such, model-generated text also contains such artifacts. To address these issues we present RankGen, a 1.2B parameter encoder model for English that scores model generations given a prefix. RankGen can be flexibly incorporated as a scoring function in beam search and used to decode from any pretrained language model. We train RankGen using large-scale contrastive learning to map a prefix close to the ground-truth sequence that follows it and far away from two types of negatives: (1) random sequences from the same document as the prefix, and (2) sequences generated from a large language model conditioned on the prefix. Experiments across four different language models (345M-11B parameters) and two domains show that RankGen significantly outperforms decoding algorithms like nucleus, top-k, and typical sampling, as well as contrastive decoding and search, on both automatic metrics (85.0 vs 77.3 MAUVE over nucleus) as well as human evaluations with English writers (74.5% human preference over nucleus sampling). Analysis reveals that RankGen outputs are more relevant to the prefix and improve continuity and coherence compared to baselines. We release our model checkpoints, code, and human preference data with explanations to facilitate future research.